Lessons We Learned From H4ck3rZ Online Systems

We have been using forums and CMS systems for long time now to run all CAT websites on the internet starting from the ancient CATReloadedt website that was based on Drupal and ending to this amazing and extensible system “H4ck3rZ Network” that’s based on Buddypress and we have faced hunderds of problems that we didn’t really spend the right amount of efforts to learn from and to solve.

We in H4ck3rZ have spent sometime thinking about that and studying how to stop falling into the same problems and how to engineer a system that is really scalable without the loss of users information that is the most precious piece of our software and we could really design a system that can fix most of our problems in a granular manner, so we can fix things as we go and with minimum users intervention. But because we are engineers and our H4ck3rZ community is engineering community, we decided to share you some of the internals and details of what we have done.

The Ancient Problems

For long long time a faced a problem that we didn’t learn from it, that’s having accounts of our members stored inside the system and we didn’t feel the problem until we faced the problem of migration from drupal to phpBB and the booming fact is that migration might not be possible and people have to register again, we also had a mailing list that had its accounts stored in it and it’s a completely isolated island and we had no easy way to check if both systems are in-sync or not. More problems was arisen when we started the foundation system because each foundation started building its own website and again we lost the user base and people started new registrations on each website independently and moreover people used several e-mails and usernames and that was really a hit because now you cannot identify people by username anymore a good example is that I used the username h4ck3r in H4ck3rZ Foundation online systems while I use the username ‘linuxawy’ in PixiCat and please don’t ask me why

Because H4ck3rZ is the largest foundation in its users base and because we used to had alot of online systems (blog (WP), phpBB forums, DokuWiki as a wiki engine and more)  we had a serious problem of integrating those things together so you can have a single identity over everything, and because we are engineers we started thinking about a generic solution that could solve all CAT identity problems, thus we started the cat-identity project and we didn’t find enough volunteers to work with us to work on that project but we decided that we should restart this project again and we will do this very soon isA.

H4ck3rZ have learned the lesson and now we have migrated to a single identity server with LDAP backend that we use to authenticate all of our web and online systems and that will include our project SCMP that we are about to launch the alpha version soon and now we you will enjoy having a single username and password to login to all of our systems and our future systems and no need to worry about that anymore, so probably we won’t lose our users base anymore

Availability problems

Because all of CAT websites were hosted on a single server, there were an availability problem and thus we had to move to another server to allow sort of high availability solution of one server failed and now we have designed a distributed infrastructure that allows us to split our services over multiple server without any effect to the user experience and seamlessly sessions will transport to the new server but we didn’t actually use it yet as we are still out of funding and we use our personal funding to run those servers.

We are happy about what we did and we now can host hundreds of websites and blogs of our users on our very own servers with a single sign on between those blogs and single identity and we wanted to share the experience with you.