# A10 cluster mele a2000 > Designing a low-cost, low-power ARM web server cluster from 16 Mele A2000 (Allwinner A10) nodes, aiming for full redundancy and remote power reboot. - URL: https://guillaumeplayground.net/allwinner-a10-cluster-mele-a2000/ - Author: Guillaume (https://guillaumeplayground.net/about/) - Published: 2012-09-25 - Updated: 2013-02-04 - Section: Computing & hacking - Source: Guillaume playground ![A10 cluster mele a2000 – photo 1](https://guillaumeplayground.net/wp-content/gallery/mele_a2000_cluster/12090002.jpg) ![A10 cluster mele a2000 – photo 2](https://guillaumeplayground.net/wp-content/gallery/mele_a2000_cluster/12090004.jpg) ![A10 cluster mele a2000 – photo 3](https://guillaumeplayground.net/wp-content/gallery/mele_a2000_cluster/12090005.jpg) ![A10 cluster mele a2000 – photo 4](https://guillaumeplayground.net/wp-content/gallery/mele_a2000_cluster/12090006.jpg) ![A10 cluster mele a2000 – photo 5](https://guillaumeplayground.net/wp-content/gallery/mele_a2000_cluster/12090007.jpg) ![A10 cluster mele a2000 – photo 6](https://guillaumeplayground.net/wp-content/gallery/mele_a2000_cluster/12090008.jpg) ![A10 cluster mele a2000 – photo 7](https://guillaumeplayground.net/wp-content/gallery/mele_a2000_cluster/12090009.jpg) ![A10 cluster mele a2000 – photo 8](https://guillaumeplayground.net/wp-content/gallery/mele_a2000_cluster/12100002.jpg) ![A10 cluster mele a2000 – photo 9](https://guillaumeplayground.net/wp-content/gallery/mele_a2000_cluster/12100003.jpg) ![A10 cluster mele a2000 – photo 10](https://guillaumeplayground.net/wp-content/gallery/mele_a2000_cluster/12100005.jpg) ![A10 cluster mele a2000 – photo 11](https://guillaumeplayground.net/wp-content/gallery/mele_a2000_cluster/12100007.jpg) ![A10 cluster mele a2000 – photo 12](https://guillaumeplayground.net/wp-content/gallery/mele_a2000_cluster/chart2-php_.png) ![A10 cluster mele a2000 – photo 13](https://guillaumeplayground.net/wp-content/gallery/mele_a2000_cluster/12100009.jpg) References: [http://armservers.com/2012/06/18/apache-benchmarks-for-calxedas-5-watt-web-server/](http://armservers.com/2012/06/18/apache-benchmarks-for-calxedas-5-watt-web-server/ "apache-benchmarks-for-calxedas-5-watt-web-server") [http://www.theregister.co.uk/2012/07/09/boston\_viridis\_arm\_server/](http://www.theregister.co.uk/2012/07/09/boston_viridis_arm_server/ "boston_viridis_arm_server") The Viridis arm server is $50K. The goal is simple, “create a low cost arm cluster able to serve the maximum http requests”: - low power consumption - full redundancy (not a single SPOF) - space efficiency - minimal cost - remote power reboot The first cluster (i will make two for a start) have 16 nodes. I have shortlist these items to proceed: - The Allwinner A10: a great arm-clone running at 1Ghz. - The mele a2000 platform provides the A10, 512Mo ram, 4Go NAND, network controller, the lowest cost … a case and a nice sata plug. All of this for $70… Crazy little platform. Thanks to [Tom Cubie](http://www.aliexpress.com/store/511685) for his support. Take a look at his own board: [Cubieboard](http://cubieboard.org/) - Redundant PSU. It should provide enough power for the 16 mele a2000: [Silverstone SST-ST42-GF](http://www.silverstonetek.com/product.php?pid=289&area=en) or SST-ST50-GF: 80Plus, 5V 25A, 12V 33A - Ethernet relay board. [Denvoki ethernet 16 relay board](http://denkovi.com/productv/41/internet-ethernet-sixteen-channel-relay-board.html). Great product. I have split the work in two case, the PSU and a sort of PDU with the relay board. The PDU needs 5V (for the mele) and 12V for the ethernet relay board. The 5V 25A could be a bit light to power the 16 nodes with disk. I decided to use this 5V for 10 nodes and convert 12V to 5V for the 6 nodes left. It will share the load properly. I’m using a simple DC-DC 12V to 5V 10A converter to do so (simple step-down found on ebay for $15). **Assembly part 1 – the PSU:** This PSU have one 12V and one 5V output. To avoid opening the PSU, i have proceed like that: 1. Short the PS\_ON (p14) and COM (p15) from the ATX connector. 2. Join 4 x 12V (yellow) from different connectors (pcie, …) ~ average power needed 12V 5A, max 12V 10A 3. Join 4 x 5V (red) from different connectors (molex, …) ~ average power needed 5V 10A, max 5V 20A 4. Join 4 x COM (black) from different connectors. I use a simple 230V – 10A power cable with IEC connectors to output to my PDU: - Neutral -> 5V - Ground -> COM - Phase -> 12V **Assembly part 2 – the PDU:** The PDU is nice and easy with the relay board: 1. Get the incoming 12V from the IEC: phase of the ethernet card power, the step down converter and the relay card. 2. split the output of the step down convertor to 6 relays. 3. Get the incoming 5V from the IEC: split the output to the 10 relays. 4. Get the incoming COM from the IEC: Neutral of the ethernet card, neutral of the step down convertor and neutral of all our outgoing Mele a2000 power cables. To do the assembly properly, i have plug the com of the step down convertor to the 6 Mele output cable but it’s not necessary. 5. Plug each phase of Mele a2000 power cables to each relay output. This is it 😉 I have now a remote power switch for the 16 Mele a2000 with a redundant PSU. **Software – Linux debian armhf:** The easiest way is to get the rootfs from [rhombus](http://rhombus-tech.net/allwinner_a10/). Build our own debian distrib is fairly simple by following this [guide](http://rhombus-tech.net/allwinner_a10/hacking_the_mele_a1000/Building_Debian_From_Source_Code_for_Mele/ "Building_Debian_From_Source_Code_for_Mele"). Some tweaks needs to be done for armhf: To cross compile from debian (unstable or testing): `echo "deb http://www.emdebian.org/debian/ unstable main" >> /etc/apt/sources.list apt-get update; apt-get install gcc-4.7-arm-linux-gnueabihf gcc-4.7-base-armel-cross` The cross compile env becomes: `CROSS_COMPILE=arm-linux-gnueabihf-` The debbootstrap becomes: `sudo debootstrap --verbose --arch armhf --variant=minbase --foreign testing mnt http://ftp.debian.org/debian` I had to work around some issues on the u-boot ([Forum](http://forum.doozan.com/read.php?6,9527)) and Sata reset ([Forum](http://forum.doozan.com/read.php?6,9597) / [Github](https://github.com/amery/linux-allwinner/issues/40)). I had applied also the work from lundman and Vayo from the github issue about the [wemac performance](https://github.com/amery/linux-allwinner/pull/39). At the time i will publish this page, i think this issues will be patched in github directly. **Software – Boot from nand:** I didn’t wish to get a sdcard for each Mele… The boot from nand is easy to achieve. Two methods: 1. Reuse the u-boot.bin from the nanda /linux directory, you simply have to put your uImage in this directory to get a boot. You should also regenerate the /script.bin with sunxi-tools bin2fex/fex2bin and replace the mac address. 2. Use the multi u-boot ([http://forum.doozan.com/read.php?6,10012,10082#msg-10082](http://forum.doozan.com/read.php?6,10012,10082#msg-10082)) I used the first one, it was before the multi u-boot became available. The multi u-boot is a far more elegant solution now. sunxi-tools allows you to repartition the nand properly to get the whole space in one single partition: `/usr/local/sunxi-tools/nand-part /dev/nand 'LROOTFS 4069376'` reboot and you simply have to put your debian rootfs in the fresh nandb formated with ext4. The rootfs is stable in the nand. I add a Sata SSD to each mele to get a fast and reliable storage synced with unison for the files i will serve. **Results part 1 – power consumption** Power consumption seems better than i expected:
Cluster without SSD (1Ghz – governor: performance – idle)280mA
Cluster with 16 SSD Crucial m4 (1Ghz – governor: performance)385mA
**Update – Oct 25, 2012** I’m now in full production with the 32 nodes (2×16) balanced around two datacenters. - the A10 is stable (uptime: 25 days for now and counting) - the load average is around 0.1 – 0.3 - my biggest fear was the wemac ethernet. Perfectly stable too. No request failure reported from zabbix yet. I’m pretty sure the ethernet relay board is useless now 😉 The total cost for the 32 nodes + 32 SSD + 2 PDU/PSU: $5100 or **$160 per node with SSD** or **$99 per node without SSD** - Mele a2000: $70 (with shipping) - SSD 64Go: $60 - PSU/PDU: $470 Mission complete.