Unless the online store is only a small component of your website you should go with an open-source e-commerce platform. We have worked with osCommerce, x-cart, zencart, ubercart and Magento. Overall Magento as a package is the arguably the best open-source free-to-use ecommerce platform available in the “market”. Why it is the best, will leave it for another article. Key advantages I see:

i. Out-of-the-box mobile website and mobile applications are available

ii. Beautiful (& fully customizable) themes available for all kind of verticals (electronics/books/flowers) are available.

iii. You get a feature rich back-end and admin dashboard. You name it and you will find it in Magento.

iv. You get number of options for custom functionality in terms of third party extensions and plug-ins.

v. It is built over a robust MVC based PHP framework – Zend

Some of the advantages mentioned above are applicable for osCommerce, zencart and x-cart as well. Having said this, Magento is not the fastest when it comes to performance and others platforms are also fairly capable, but overall Magento is a very nice package to start with.

When should I build an e-commerce application from scratch and not use a platform?
Almost never. Because it is a myth that one has to compromise on the quality or speed or customizability if he/she uses an existing platform to build the online store. Building a full-fledged online store from scratch is not a trivial task. An online store is not just a shopping cart module. These days, there are so many backend modules and front-end features in any “respectable” online store that it can take you months and maybe a year to build something that will come close to what a fully tested, feature rich and widely used e-commerce platform such as Magento can provide. Moreover, you will have to put in all the effort to make your store render perfectly on tablets and smart-phones. Why do that when somebody has already done that for you. An online store built over a popular platform requires much lesser maintenance as regular code updates are available in the form of new releases. To summarize the key advantages of using an e-commerce platform are:

• Much lesser go-to-market time
• Tried and tested
• Regular updates and addition of new features
• Mobile friendly sites get created without too much of additional work
• Much lesser investment. The cost saving can be used on marketing the site.

When you are getting into the online retail business you should focus more on core business tasks like building a strong base of suppliers, logistics, variety of items available in your store, explore cross selling opportunities, marketing and most importantly, delivery & customer support. While it is important to choose a scalable and flexible platform for future needs, planning too much into the future is not the best idea at the starting-up stage. If your online store is able to attract good numbers of customers who are doing actual transactions then it is more likely that you shall also be able to raise investment. At that point of time you can even go for a custom-built e-commerce site if you feel that it would not be possible to scale the existing site built on a platform such as Magento for the huge amount of traffic your site is expecting. Till the time you have validated your idea and business model you should stick to an out-of-the-box solution with some customizations as the case maybe.

Lets not solve for problems that don’t exist just yet.

It is required at this stage to mention the CAP Theorem, which says that any distributed computer system can provide at max two of these properties: Consistency, Availability and Partition tolerance. Consistency says that the data will be consistent throughout the database system. Availability can be thought of as the up-time of the system and Partition tolerance means that if a few messages are lost, the system should continue working. Relational databases guarantee Consistency and Availability, but at the cost of speed. There are applications where a minor loss of data or delay in updation of data is no big issue. These applications don’t require the overheads of consistency and can give it away for greater speed. Here we welcome NoSQL, popularly expanded as Not Only SQL, which guarantees Availability and Partition tolerance. Most of the popular systems also talk about eventual consistency, which means that over a long period of time, due to all the changes made in the database, the data will eventually become consistent.

NoSQL covers a wide variety of storage systems that do not use SQL as their querying language. Majorly, they can be classified as Key-Value systems, Document based systems, Graph databases and BigTable based systems. These are non-relational systems, don’t go by fixed schemas and do not use joins. The reason for their existence is the need of scalability. While a MySQL system will require faster hardware for scaling (vertical scaling), a NoSQL system can just add a new cheap machine to add to the power (horizontal scaling). Relational systems implement the ACID properties: Atomicity, Consistency, Isolation and Durablity. NoSQL systems implement BASE: Basically Available, Soft state, Eventually consistent). Their potential can be better realised after knowing that web giants such as Google, Twitter, Facebook and FourSquare have migrated most of their systems to some form of NoSQL databases.

NoSQL however has its own set of issues. Being relatively new, not all implementations are truly reliable. Proper documentations are not available for all systems; all that’s there are some blog posts and forum discussions, and one may have to dive into the source to understand how to get things done. However, most of the popular implementations like MongoDB, Cassandra, CouchDB and HBase are devoid of these issues and are being used in many production applications.

The question is, is it suitable for you? Take the case of Twitter. If a person’s timeline doesn’t show a tweet just as it was added or doesn’t show one of the followers in the list, it hardly matters. Twitter can compromise with consistency for speed. However, if a Walmart system shows the day’s sales figures as different from what they actually are, it can create a lot of problem. They have to have consistent data in any case. So, it just depends on what you are planning to make.

- Drupal7 includes a number of usability improvements for site administrators to make their job easy. This is a big step forward as this was a common complaint from Drupal detractors.

A number of things have been done to achieve the same. (i) edit-content functionality is much easier to use now with better text editing modules and features (ii) the admin dashboard is completely customizable now (iii) It is much easier in Drupal7 to find the task correctly

- Drupal has taken a lead and implemented RDF in Drupal7. This should be a clincher for those who are looking to implement the semantic web on their websites.

- There are improvements in security aspects. Once the frameworks bring in new securiy features, the hackers attempt even newer techniques and then the framework architects respond by bringing in new security features in new release. Since Drupal7 is a major release so new security features were expected.

- The default theme is much more attarctive and looks modern. The theme for admin console is also brand new.

- Almost every application built in Drupal these days uses CCK [Content Construction Kit] module. In Drupal7 CCK has been added to Drupal core and thus enabling much tighter integration. Second insllation profile can be used as a module to make peripheral changes in user profile.

- One would always expect the customizability of look and feel in any CMS. Drupal never imposed its own layout and UI and always had a good theming layer. Drupal7 goes further ahead in improving the theming layer concept.

- The installation process of Drupal modules would be much more simpler now. One can even do it with UI in a couple of clicks. Not everybody likes FTP and command based installation. Right??

- Drupal7 ships with JQuery1.4.4. This version of JQuery is faster and provides excellent features to developers such as $.proxy().

- Last but not the least: Scalability is an important aspect while choosing the right framework for your application. Drupal has never been the preferred choice for performance fanatics. But we know some very high traffic websites built on drupal such as The Economist. Drupal has worked on the performance and scalability aspect in Drupal7 and has achieved significant results on this count. New optimization and caching techniques implemented in Drupal indeed shows results.

We at Mobicules has always focussed on performance and scalability aspect and have published some information articles on our tech-blog. You can read more about performance and scaling and Choosing a framework/platform for a social network that needs to be 10 million users ready on our blog.

As a Company or even as someone who wants to be host to web services, this is definitely something you want to be familiar with.

Dynamic, on-demand-scaling has been and remains the biggest advantage of Amazon EC2. You can use the Amazon EC2 web service APIs to start, monitor, terminate, etc. as many virtual machines as you want. This means you can scale your application to as many servers as you need on the fly. If you do the resource (web servers) planning to handle the potential “peak load” then for majority of the time, you are likely to use only 10% – 20% of your computing resources. The other 80% is sitting there “just in case”. It’s obviously not a great use of resources and/or money. This is where EC2 scores. The resource allocation on Amazon EC2 can be done dynamically and programmatically i.e. you can “reserve” the servers on the fly as per the traffic load. You don’t have to worry about scaling. Let Amazon handle that for you.

Another, big advantage of EC2 is Amazon’s brand name. To quote the Chief Geek and Co-founder of a SaaS company – Grexit, Nitesh Nandy: “”AWS’ compliance with Type II SAS 70 requirements and PCI data security requirements made it a very good fit for us to store our data and code in a virtualized environment. For SAAS applications that deal with a lot of sensitive customer data, AWS provides a great infrastructure which is secure, and also offers dynamic scaling of compute and storage capabilities.”. (Disclaimer: This articles is primarily based on the inputs gathered from EC2 users Niraj Ranjan Rout and Nitesh Nandy, Founders of GrexIt Inc.)

Amazon EC2 has its share of cons as well.
- Amazon web services are costlier than the other options such as Linode. But If you go for a reserved instance, the cost is less, on demand ones cost more.
- No persistence in instances. If you want persistence you will have to get EBS. On EBS if you have lots on I/O operations, you get charged a lot. EBS is good on machines that host database and other data as its a real time backup.
- There is a learning curve to ec2 for sure. But, once you get past the learning curve and get comfortable with the idea of putting data only on machines which have EBS and using other machines as pure compute units, Amazon is good to work on. I think that is the key part of the learning curve (Plus making insances, images etc.).

Bottom-line: If one primarily needs dynamic scaling then it makes more sense to go for ec2, otherwise EC2 is not the best option and not the best value for money for sure.

So what is this semantic web all about? The Semantic Web is all about enabling applications / documents / links in the whole web talk to and exchange data with each other. Imagine if you were to plan for your vacation. In the Semantic Web ideally, your calendar, Maps, weather forecast website and bank accounts will be able to talk to each other. There will be a robot sitting at the back who will only receive a single command from you saying that you want to go for a vacation and then it does all the job and planning for you. The robot will take your appointments from your calendar and then decide possible days when you are free. It checks for possible destinations on Google Maps which may be covered in that duration you are free and then automatically checks for hotel reservations available over the web. The robot also will also be capable of checking if the weather is suitable for travel and then access your bank account to check for funds required to book an affordable hotel. This example might be too farfetched at this point of time and many of us may not even want that much automation. But technically speaking to make these possible different apps should be able to talk to each other even if these are written in different languages/frameworks/technologies. The universal standard for making different apps talk to each other to enable the path for web3.0 is not yet finalized and its a matter of debate in the web community/leaders.

WordPress has had support for multiple websites for a long time in the form of WordPress MU. Starting WordPress 3.0, launched in June 2010. WP MU was integrated with WordPress. This means, that a single installation of WordPress can be used to manage multiple websites. It can allow users to start their own blogs within the site, each with its own theme, plugins and set of permissions.

While earlier only Post and Page were the supported content types, the newly launched Custom Content Type Manager plugin gives you all the freedom relating to content types, and is a serious answer to Drupal’s all powerful CCK. With an option to create custom content types, WordPress can finally shed of the tag of being a primarily blogging platform.

Another feature that it has recently borrowed from rivals Drupal and Joomla is custom taxonomies. Taxonomies allow better arrangement of content and better interlinking of various posts. They are similar to the pre-existing tags and categories, only more powerful as they are totally custom and give the user much more control over what he wants to see and how. For example, in a car listing, you may have a field for Seating Capacity. Using taxonomies, you can connect different cars on the basis of their seating capacities.

What makes it a complete CMS are its e-commerce plugins. Although they are not as mature and feature rich as the popular ones in Drupal and Joomla, they do their job pretty well. Plugins like WooCommerce, Cart66 and Shopp are worth mentioning. All of them have multi-site support provide all kinds of features like product configuration, shipping details and tax options.

Although most of the features provided by WP are available in other CMSs too, there are quite a few things that set it apart. Compared to other popular options like Drupal and Joomla, WP has a relatively smaller disk footprint. Setting up WordPress is very easy and takes less than 5 minutes. Its administrative panel is very intuitive and anybody can start using it easily, even if it’s their first time. In newer versions, the admin panel comes with help tips, which further ease out the usage. Among all the CMSs, WordPress has the largest developer community which is constantly working toward delivering a better solution. These things make WordPress the CMS of the future.

Whether you choose to go with a CMS or an MVC depends on what you are building and what kind of effort and money you are ready to put in. A CMS will work for you out of the box. It will have the basic functionality that a portal needs and additional features will be available as add-ons or plugins. With popular CMSs like Drupal and WordPress, you can expect to find a plugin for any regular feature that you want to incorporate and you won’t have to write a single line of code. The problem with a CMS begins when you want something very specific which is not available as a plugin. Suppose the user login and authentication systems is different from what comes out-of-the-box with the CMS then, it’s reasonable to assume that you will not find a plugin that suits your needs and you will have to modify the existing plugin or write your own. You may write a plugin for the CMS but with the constraints placed by a CMS, depending upon the effort required, it may be a better option to build your own application from scratch. An MVC framework like Symfony will give you all the tools you need to build up your application and you can make the application exactly the way you want it. If you have a lot of business logic to implement, a CMS is not for you. Using Symfony will give you complete control of things such as data storage and UI, it will allow you to separate your business logic from the presentation layer and also, it will let you use the latest in technology.

If you have decided to go for a CMS, you still have a lot of options to choose from, the popular ones being Drupal and WordPress. Both are great in their own way. WordPress powers almost 15% of the world’s top websites and this figure alone speaks volumes. It is really quick and easy to configure and can get you a multi-user blogging platform ready in under 5 minutes. With thousands of free and paid themes and close to 15000 plugins, you can very well get a site close to what you wanted. Other things that work in its favour are easy administration, low maintenance costs, large community, frequent updates and good SEO out of the box. While it is mainly a blogging platform, it has been successfully implemented in News portals, image sharing and other such fields where the content types are limited. For that matter, WordPress allows only two basic content types- Post and Page. So you can do a lot of stuff on this platform, but it cannot provide you with everything that you may want. Here comes Drupal. It powers close to 2% of the world’s top websites. It is heavier than WordPress, not as intuitive and is relatively costly to maintain. So why would you use it? Because it can provide you with everything from forums and user networks to fully functional shopping portals and payment gateways as add-ons. It allows you to create your own custom content types, and also design the view of your pages. It has a very active community and updates are regular. So, if you are not planning on writing a single line of code, you can still have an e-commerce site or a social network ready, and obviously, you can write your own add-ons .

So, in a nutshell, if your website is something very specific to your needs, don’t go for a CMS. Use a Framework such as Symfony or CodeIgniter. If your project is something generic, using a CMS will save a lot of effort. Go for WordPress if you have simple content types. Large applications with complex functionality are better off on Drupal.

Before scaling, make sure you have optimized your existing code to work properly with whatever resources you have. Here are a few options that you should look at first.

Presentation Layer
Improvements in this field are among the easiest to implement and are seldom the major culprit for a poor user experience. You can use a browser add-on like YSlow or PageSpeed to see the performance status of your application. An excellent article is available here outlining the issues and their resolutions.

Server
You can start by using a profiler like X-debug to check the areas which need improvement or are slowing down your website. You may be running unnecessary loops and slow database queries and perhaps queries in a loop. The profiler will help you identify these and take appropriate action.
Check the apache logs to see if there are any problems. See if you are logging too much or unnecessary data. Are you including any unnecessary apache modules – do away with any component that is not required. Here is a more comprehensive guide for tuning apache performance.
Try a linux command line tool like top or procinfo to take note of apache performance and resource it is using.
Next, try using a caching program like APC or X-cache. This would improve performance significantly.
Move to a dedicated server if you have not.
Use a CDN – they will reduce load on your server by taking care of the static data and leave your server to handle the dynamic part.

If you have implemented above and think you still need to scale to multiple servers, you have made the right decision. To begine with, you must have answers to this question: For how long do you need to scale and how many servers would you be needing?

Spending on hardware yourself should be your last resort as they are expensive. Also, simply increasing the processing power and RAM of your single server may not be the best of the ideas. It is more feasible to deploy across multiple smaller servers and employ load balancing to distribute load. This way is cheaper, easier to extend and removes SPoF problem.

But first try cloud solutions – there are various services like Amazon and Rackspace that provide many flexible options. This would be best to go for as you do not have to worry about the hardware setup and maintenance. Some may also provide you with autoscale on demand option.

If these do not work for you, then scale to more servers.
You can use a tool like rsync which will ease your process of getting your files on each server and maintaining same state across all of them. Alternatively you can also use Subversion (SVN) to update all servers separately. You can also use a file server like NFS for this but they will also become an SPoF. Find more info here.
You can implement load balancing directly from Apache using mod_proxy_balancer or you use round robin dns.
You should also employ proper logging methods so they easy to manage across all servers. As an example you use a dedicated program like syslog for this.

When scaling, session management is an important aspect to consider. If a user is directed to different server on each request, he will be continuously logged out. You can tackle this by storing session data in MySQL or using memcache to handle the session.

Database
Keeping a check on your db performance is very important.

First make sure you are using the right storage engine for your requirements. These can be done on table level, so if the table is accessed for mostly reads, you should use MyISAM and if it write intensive, use INNODB. You should also use MEMORY tables for frequently accessed data as this would be significantly faster.
Check the slow query log frequently to see if there are some specific queries that are destroying the otherwise good performance.
Go for batch inserts wherever practical as it would be faster than multiple individual inserts.
Make sure you are using MySQL query cache optimally and the settings are right. Read here for more info.
If your database houses a lot of data, you can try making partitions.
If you need further improvement, try hosting your db on a separate machine which is fine tuned for serving database queries. Mysql Accelerators are available that will take care of most of the scaling part.

If you still find that you need to scale to multiple servers on your own after following the above steps, you may be right.

If your application is read intensive, you can distribute read load among slaves. This will be fairly easy using mysql replication.
Distributing writes is more difficult. You can employ sharding to distribute write load among multiple servers. Here is a good article related to this topic.
You can further speed up your database by using memcache. This will make the db queries considerably faster.
Using clusters may help if you need high availability and performance, but they may not be the solution for distributing write load.

Conclusion
This is just the tip of the iceberg and what steps you take to improve an application’s performance is very specific to your application. Scaling an application comes with a price – either in terms of installation and running cost or in terms of manageability and code changes or enhancements.
This blog has been written to serve as a guide for options available to you before and when you decide to work on improving performance or scale your app.

Your comments, corrections or improvements are welcomed.

Symfony is an Open-Source PHP Web Application Framework which Mobicules started using for large and complex web applications and is now at the forefront of Symfony development. Here is the brief synopsis of some of the applications we have developed in Symfony 1.1 and 1.2:

• www.voyavo.com
• The backend dashboard for AGame
• The Innovative Recruitment Application MyJobLinx in use by Adobe Corporation.
• WushuPlayer.com
• Backend admin panel for iPhone application Yo-Momma Battle and Penance

After developing and maintaining a variety of web applications in Symfony 1.1 and 1.2 we are now geared up to adopt Symfony2 for future development. Based on an innovative architecture, Symfony2 is fast, flexible, and easy to learn. It allows developers to build better and more easily maintainable websites with PHP. Following are some of the reasons why we are so excited to embrace Symfony2 for future development:

High on performance and easy on resources: Symfony2 has been developed primarily keeping the performance in mind. It is as much as 3 times faster than its previous versions and Zend which is another PHP framework based on MVC Paradigm. At the same time Symfony2 consumes very little RAM for the same or faster performance than its predecessors.

High on productivity: Symfony2 provides many useful tools for development and debugging. Here are some examples:

The Web Debug Toolbar: It appears at the bottom of every page in the development environment and provides access to lot of information instantaneously which provides access to lot of information which is extremely help for debugging.

To quote directly from the Symfony’s official website:

“Symfony was the very first framework to propose such a web debug toolbar to ease debugging. Since then, numerous other web frameworks have followed the same path and you can find web debug toolbars for Zend Framework, Django, Ruby on Rails, Rack, and CakePHP.”

The Web Profiler: The Symfony2 profiler collects information about web requests and stores them for later analysis. The Web Profiler is a visualization tool for profiling data that can be used to debug the source code and enhance performance. It can also be used to explore problems that occur in production.

Separate Environments for Development, Test, Staging and Production: Symfony comes bundled with those environments by default. A development environment refers to a server tier designated to a specific stage in a release process. Here is the direct example quoted on http://symfony-reloaded.org:

The development environment must be optimized for the developer (the web debug toolbar is enabled, nice exceptions are shown on the screen in case of an error, and all the details of a request are logged to ease debugging). But the production environment should be optimized for performance and the user experience (errors should be masked behind a friendly page, and an aggressive cache layer must be activated). Creating a new environment in Symfony is a matter of creating a new set of settings.

In addition to above mentioned development tools Symfony2 also provide helpful error messages and useful logs. More details can be found at: http://symfony-reloaded.org/tools

To summarize Symfony2 is fast, flexible, extensible, borrows the best concepts from Symfony1, Django, Spring, Ruby on Rails and last but not the least it is built for developers.

Read our earlier blog entry on Symfony here

To receive a quote for Symfony development services please write to us at info@mobicules.com to get the process started.

E4X stands for “ECMAScript for XML” and ECMA in turn stands for European Computer Manufacturers Association. ECMAScript for XML (E4X) is a programming language extension. It adds native XML support to ECMAScript. Here ECMAScript ActionScript, DMDScript,JavaScript, and JScript.

ECMA is the organization standardizing Javascript. ECMA has also developed standards for quite a few stuff apart from Javascript e.g. C# language, Optical disks, Magnetic tape, Data compression and communication etc .

E4X is available to both Firefox and ActionScript developers that turns XML into a native data type on par with strings, arrays, objects, and regular expressions. Worth to mention its not the DOM equivalent of XML but is a distinct entity itself which has its own set of methods. E4X is implemented in SpiderMonkey (Gecko’s JavaScript engine) and in Rhino (JavaScript engine written in Java).

Why Use E4X?

There is a basic question which arises out of curiosity needs to be answred while addressing new technology.Why work with E4X when you can do the same thing with JSON objects (keeping in mind that both E4X and JSON objects are “part” of JavaScript, though JSON is an expression of the JavaScript Object object)?

JSON is marginally more compact to encode— than for attribute intensive XML or element heavy encodings. But this advantage disappears if the resource is gzipped as for both the cases we are storing labels and associated values. Thus if transmission is concerned then gzipping the content will occur as a matter of fact and the speed becomes insignificant comparison. Apart from this some very interesting work is going on with EXI (i.e. Efficient XML Interchange) that to at W3C level that is trying to make EXI more compact than even compressed JSON.

Moreover it is very easy to use and user friendly as for an example we can define the XML object variable just any other variable such as Date, Array etc. using new operator.

Such as

var myXmlObj = new XML();

var myDate = new Date();

The most significant difference between working JSON and working with XML using E4X is that the latter is an example of array programming.

Things like “html.body.p” aren’t possible to express in most programming languages. The fact to be noticed here is above expression in E4X and JSON means something completely different.

In E4X “html.body” returns a trampoline object (XMLList), which asks all the elements in it for their “p” children, where as working with JSON, “html.body” returns one object or a list of objects. If it’s a list we write loop to do something with children. Just getting the text contents of all p will require 2 nested loops. Of course the code behind E4X uses loops but the point is we don’t have to.
Creating E4X Objects

Creating E4X object is very easy …

e.g.

To load an XML doc

var xmlDoc=new XML();

xmlDoc.load("note.xml");

document.write(xmlDoc.body);



The above will print the content of body tag from the note.xml doc



We can access any XML node using “.” (dots)

e.g.

var myXmlVar =  Mr XMr.Y

document.write(myXmlVar.to);

Above code will print “Mr X” 

Key features of E4X include

• ECMAScript for XML specification defines a new set of classes and functionality for working with XML data.
• You manipulate XML data with well-known operators, such as the dot (.) operator.
• Use the @ and (.) operators not only to read data, but also to assign data.
• Using E4X functionality is much easier and more intuitive than “walking the DOM”.
• E4X can be used for creating templates for dynamic content.

If one adds a few common purpose functions (especially along with the convenience of JavaScript 1.8 expression closures), the templates can function more dynamically, offering the power and readability of templating languages such as Smarty for PHP (though admittedly without the currently wider cross-browser support of XSLT or the strictly-XML approach of PHPTAL or See through templating).

E4X Support

• Firefox1.5 and above supports E4X.
• Currently there is no support for E4X in IE,Opera, Chrome, or Safari

Limitations of E4X

• It is not currently possible to access a DOM object through E4X.
• E4X doesn’t support parsing XML declaration

http://en.wikipedia.org/wiki/ECMAScript_for_XML