Who is the product? Who is the customer? Do you even care?

Facebook, how big?

Facebook has obtained more than $500M from investors to grow facebook.com to the size it is today, and has not taken a single penny from any of its users. In order to pay back the investors for their extraordinary risky investment, the investors must be looking for something like 10x cash back.  That’s more than $5 billion! Of course it’s currently valued at over $50B. Just hold that thought for a second.

Now a second thought: billions of people can use email and not have to be part of one, single, organisation. How can that be?

Email is essentially a protocol. It’s called SMTP and is described by various RFCs. Any server that supports the SMTP protocol can advertise its MX record via DNS and receive email for that domain. Any client that ‘talks’ SMTP can send email to any SMTP server (it can reach). In fact, the SMTP client (or email client) can talk to its local SMTP server which will then forward on the email to its final destination.

This is, of course, a distributed system.  Due to an open protocol anybody can set up an email server and play in the big email ecosystem. Of course, the original inventors of the SMTP protocol didn’t envisage SPAM as we know it, and thus it was designed for a naive, friendly, co-operative world, where email users wouldn’t spam each other. i.e. academia.

Facebook is set up as a business. It has its customers’ interests to serve so that it can be a profitable company and return its investors money and provide a return for its financial stakeholders. The problem is, is that Facebook users are not the customer. They are the product that is sold to the actual customers who (I suspect) are advertisers. Thus, Facebook’s values aren’t necessarily aligned with their users, which means, almost inevitably, privacy is not Facebook’s key concern.

So if with Facebook, the users are the product, what are they actually selling? The social graphs its users create, along with the logged minutiae of the their lives, could just be the product that Facebook is, and will continue, to sell to advertisers. Your privacy is Facebook’s product. Your social graph (in theory) has value to advertisers. Do you want to exchange your privacy to multitudinous corporations for free access to, well, Facebook?

But the main problem with Facebook is that, in order to do Facebook with somebody else, you have to have an account at Facebook.com. It’s a closed system. Notice the difference to email? I don’t have to have an account at (the fictitious) email.com to send emails to other people. That would be absurd!

History has a habit of repeating itself. Remember CompuServe? AOL? MSN (pre-internet)? CIX? These were all silos. CompuServe had special pages only subscribers could see. Of course, they all went the way of the dinosaur, or were heavily modified, because the Internet was more useful. And Facebook is simply a better CompuServe or AOL.

Still, you may ask, “what’s wrong with a better AOL or CompuServe?”

Proprietary Silo vs Open Data and Open Protocols

Before answering that question, let’s consider what the alternative to Facebook or Twitter would be like. By way of analogy let’s look at Twitter vs status.net.

Twitter and status.net basically do the same thing: they are broadcast micro-blogging systems that let you send the equivalent of an SMS over the Internet to your followers. The best known example of status.net is identi.ca.

If you want to be part of the Twitter-verse you have to get an account at the sole provider of twitter-ness: twitter.com. Every tweet you send goes through twitter.com, is stored there, analysed and also provided to other (rich) organizations via the Twitter fire-hose. Not a great deal of privacy there, apart from the notional privacy that you can ‘protect’ your account. It’s still going through twitter.com.

status.net is like Twitter except that it is both an Open Source project and a Open set of Protocols. The difference is that status.net is like email; it’s a protocol that anybody can implement. You could subscribe to any status.net server and still be followed by any other status.net user in the world. Therefore, your tweets, or rather dents would only go through the distributed servers, the same as email today.

Another advantage of a distributed network is that it’s more resilient to failure. Twitter went down on Christmas day because every tweet goes through twitter. Email didn’t go down, except maybe a few distributed nodes did – but email didn’t fail en masse.

So back to Facebook and silos? I definitely want to put an activity stream (or lifestream) on the web. I currently use twitter for that because so many of my friends do. I also want to be able to put the odd photo up, publish a free/busy calendar, and enable old friends and new ones to find me and get in contact. But I don’t want Facebook to own that information. I want to own it. I want it under my own control, possibly in my own appliance running somewhere on the net. A distributed system that talks to other systems to exchange the data all under my control, with my privacy settings which won’t suddenly change because an over-arching corporation needs to sell more of my privacy.

Of course, it is coming, and there’s even some competition in the space. onesocialweb and Diaspora are both trying to solve the Facebook problem. The Freedom Box project, inspired by Eben Moglen, is also trying to work in that space. These will be tools that work in a distributed fashion.

When will it displace Facebook and Twitter? I think the jury is definitely out on that one; AOL, MSN, Compuserve, MySpace: the Internet is littered with the corpses of previously all-mighty corporations that owned the space.

Personally, I want Facebook to fail. I want a future where people control their own information. I want the semantic web and the power of pull, not the push-web. I want my appliance with my data and my control and I’m prepared to pay for it. I guess I’m going to have to wait a bit!

I’d like a new phone. Unfortunately, virtually every phone released these days running Android 2.2 (Froyo) also runs some kind of customised interface. Motorola has Motoblur, HTC has Sense, and who knows what Samsung has, but it isn’t stock Android. Is it so much to ask to have the stock Android experience? Apparently, it is.

I’ve seen rumours that Google is pushing handset makers to use stock Android 3.0 rather than put their own interface on it. Here’s hoping. So, since my phone is almost out of contract, I thought I’d root it (which voids the warranty), and put CyanogenMod 6 (CM6) on it, which is basically Android 2.2 (Froyo) with a few enhancements. CyanogenMod has done some incredible work and it is well worth checking out whether your Android phone is supported.

Now I was quite nervous about messing with my phone. It has some paid apps on it and I’ve spent 18 months with it, customising it to my requirements. However, I’d heard such great things about CM6 that I was really looking forward to it. And, I must say, it is so much better that 1.6. Seriously, if you have an HTC Magic, then just read through this post, read the linked sites, and go for it!

Now the following is specific to UK HTC Magic devices, but the links indicate what to do for other phones. The steps are:

1. Check your phone really is a HTC Magic Sapphire PVT32B
2. Root your phone
3. Backup your applications/data (although I could’ve missed out this step)
4. Backup your entire 1.6 stock ROM
5. Backup your whole SD Card
6. Install CM6
7. Install Google Apps (so you can get Market, GMail, etc.)
8. Re-configure the device for UK Vodafone
9. Re-install your Apps from Market (and restore your data if you need to).

Now, I don’t really keep any data on my phone that isn’t in the Cloud somewhere. It is either on Google, Dropbox, or I copy it off the SD Card and stick it on my laptop. However, you may need to do Step 9. “restore data” if you have kept a lot of data on the phone. I didn’t have to because I use Google for my mail, contacts, calendar, etc. which simplifies the process. This was a conscious decision about 6 months ago to simplify my life. But I do have automated backups of all my data on Google.

Assumptions

• You know how to mount/read your SD Card in your phone in your OS
• You don’t panic if things don’t go exactly according to plan. You might need to search forums if your experience doesn’t exactly match mine. I certainly had to because mine didn’t exactly match some of the other tutorials.
• This is not a tutorial; it is a summary of what I did to get my phone running CM6.
• It’ll be useful if you do this where you have WiFi because when you wipe the phone it will (temporarily) not have the settings to access data over 3G on the Vodafone network.  It’ll also be quicker!
• I may have forgotten a few items whilst writing this. I had the following unusual options enabled on the phone:

1. Settings->Applications: “Unknown Sources”: ticked
2. Settings->Applications->Development: “USB Debugging”: ticked

Step 1: Check your phone is the same as mine!

My phone is a UK spec Vodafone HTC Magic 32B with 192MB RAM. I verified this! In the USA this phone is also called a MyTouch 3G/ION.

To check your phone is the same as mine you need to reboot your phone into fastboot. Shut your phone down and then hold down the ‘Back’ button whilst switching the phone on. You’ll then get a display which has this on it:

SAPPHIRE PVT 32B SHIP S-ON G
HBOOT-1.33.0004 (SAPP10000)
CPLD-10
RADIO-2.22.19.26I
Apr 9 2009, 23:30:40

The key part is the ’32B’ above which I’ve highlighted. If you don’t have the same as mine then you’ll need to follow a ’32A’ or other set of instructions. (Thanks to Ralph Slooten for the idea on showing the fastboot).

Step 2: Root your phone

This is really simple.  Search market for Universal Androot. Install it and launch it. Press “Root ”.  After a short while your phone will be ‘rooted’, which means that applications can (with permission) access anything on the phone’s filesystem. This is important as you can’t install the other tools without.

Note that you will void your warranty on the phone by doing this. Mine’s so close to being out of contract (5 days), and I’ve had it 18 months, that this didn’t worry me. It might worry you though!

Step 3: Backup your applications/data

Now you’ve rooted your phone you can back everything up on it. Search for ‘Titanium backup‘ on the Market. Install it, and run it. It will ask for root privileges which you should allow and tick the box to always allow Titanium to have root privileges.

Then you need to click on the ‘Problems?‘ button and do the upgrade for BusyBox. If you don’t then Titanium Backup won’t work.

After that, use the ‘menu‘ button, select ‘Batch‘ and click the ‘Run‘ button next to “Backup all user apps + system data“. This will make a full backup of all your user applications (but not the current stock 1.6 ROM) and all the data on the phone to the SD Card.

Now mount your phone to your computer: plug in a USB cable, connect it to your handy laptop, wait a few seconds for your phone to recognise the link, pull down on the status bar and select the little USB symbol and connect it to your computer.  Then copy the TitaniumBackup folder off the SD Card onto your computer.

Step 4: Backup your entire 1.6 stock ROM

Next we make a complete backup of your current Stock 1.6 ROM, data, apps … basically everything. Get this right and you’ll be able to return your phone to its current state.

So why do Step 3 at all? Mostly because:

• I’m paranoid
• It’s a useful thing to do for the future too.

To do this step we will be using ROM Manager by the @koush. This really is an amazing piece of software and makes it very easy to try out custom ROMs.

I used this guide to perform the backup. Please go and read it for background information and to familiarise yourself with the screens. The essential steps are:

1. Install ROM Manager – search for it in Market.
2. IMPORTANT: Work with a fully charged phone.
3. Unplug the USB cable.
4. Install the Recovery Image using the top option in ROM Manager. The PVT 32B (i.e. the Vodafone UK HTC Magic) is known as a MyTouch 3G/ION to ROM Manager.  Do not select the “HTC Magic” option or you will flash the wrong recovery image.
5. Then select “Backup Current ROM”.
6. After the phone has rebooted and backed up the ROM, it will boot back into your normal stock Android 1.6.

Okay, now your current ROM is backed up to the SD Card.

Step 5: Backup your whole SD Card

I said I was paranoid didn’t I? There is a small chance that the SD Card will get corrupted when flashing a ROM so it’s prudent to back-up the SD Card:

1. Mount it on your laptop/computer.
2. Copy everything on the card.  I put it into a ZIP file.

This copies the current ROM you backed up in Step 4 onto your laptop/computer as well.

Step 6: Install CyanogenMod 6

Okay, finally you are reaching the exciting part, but I just want to make something clear:

This step involves wiping everything from your phone, just like a factory reset. The SD Card should be okay, but all your applications and data are going to be wiped from the phone. You will be able to restore your paid for apps from Market, but there won’t be a record of what you have installed.

I would (and did) at this point make a note of what apps I had installed so I could re-install them afterwards.

Next I read [Complete Guide] How To Flash A Custom ROM To Your Android Phone With ROM Manager + Full Backup & Restore.

Note: The stock Vodafone 1.6 Android seems to re-flash the recovery image to a stock image after each re-boot. This one caught me out. So you need to re-flash the recovery image after each reboot into the stock OS. The CyanogenMod ROMs (and other’s apparently) don’t do this.

Therefore, as we have already rebooted into 1.6 Android (to do the ROM back-up) we need to convince ROM Manager to re-flash the Clockworkmod recovery image. Unfortunately, at the time of writing the ROM Manager incorrectly reports that it is flashed: it isn’t and you can prove it by trying to reboot – you’ll end up with a large ‘!’ warning triangle on recovery boot (hold ‘home’ button when powering on).

To convince ROM Manager to reflash the clockworkmod recovery image, just scroll to the bottom of the ROM Manager app and install the aron_RA image. After that is installed, re-install the clockworkmod recovery image, but using the option at the top again. Now you are ready to flash the CM6.  (Thanks to this thread at clockworkmod forum.)

Now we pick up on the instructions at Upgrading from CyanogenMod 4.2 to CyanogenMod 5/6. We’re not really upgrading, but it works well.

If you bought the ROM Manager application you can probably browse for the ROM. I didn’t because I wasn’t sure if it was going to work. I am going to buy it now because it does such a great job and I want to support its development.

So this is how it is done manually:

1. On your laptop, download the latest (in my case CM6) ROM. This is the CM6 ROM in the section “CyanogenMod for the HTC Dream & Magic” on this page. The MD5SUM for the ROM I downloaded is: 35b6603227aea922c28284ef8d269015 and the filename is: update-cm-6.0.0-DS-signed.zip
2. Also download the tiny MDPI version of the Google Apps for CyanogenMod 6. Again the MD5SUM is d21e9a3597b7f03f9e623c8e5b9a9caf and the filename: gapps-mdpi-tiny-20100917-signed.zip
3. Next copy these two zip files over to your phone’s SD Card. Anywhere is fine, but I placed them in the top-level directory in the SD Card.
4. Ensure the phone is fully charged.
5. Disconnect it from the laptop – i.e. unplug the USB.
6. Run ROM Manager and select “Install ROM from SD Card” (see images on this helpful page!).  Select the update-cm-6.0.0-DS-signed.zip file.
7. Ensure you tick the “Wipe Data and Cache”.  There is no need to select/tick the “Backup Existing ROM” as you have already completed that step above.  Incidentally, the reason I split it was so that I could copy the backed up ROM off the device before flashing the new ROM.
8. Click “ok” and it should flash the ROM.

The phone will reboot into the recovery image, flash the new ROM, and then reboot again. It should now be running CM6.

Okay, now you have CM6, we still need to put the Google Apps on the device and get it back onto Vodafone.

Step 7: Install Google Apps

This is so you get Gmail, Market and the other Google Apps.

1. Run ROM Manager and select “Install ROM from SD Card”. This time select the gapps-mdpi-tiny-20100917-signed.zip file.
2. Ensure that neither of the two tick boxes are selected (“Wipe Data and Cache” nor “Backup Existing ROM”)
3. Click OK.
4. The phone will reboot and install Google Apps.

When the phone reboots this time it will go through the Google Apps set-up.

I have two Google Accounts that I wanted to use with the Phone. One is a Google Apps account (i.e. a paid for hosted account on Google for my domains), and the other is a typical googlemail/gmail account. As Market can only be used with your ‘regular’ Google account, I did the set-up with my regular account. This means that Market will recognise your paid apps.

The Froyo 2.2 Gmail, Calendar and Contacts programs are multi-account aware and can be configured with both the regular account and the Google Apps account. This essentially solved my problem with 1.6 (single accounts only).

So my recommendation is to set the phone up with your regular Google account and add the Google Apps account in the Accounts section. Note that Reader isn’t multi-account aware and will only work with the primary account on the device.

Step 8: Re-configure the device for UK Vodafone

Navigate to Settings->Wireless & networks->Mobile Networks->Network operators and select “vodafone UK”.

Then go back to Settings->Wireless & networks->Mobile Networks->Access Point Names and select “Vodafone UK Contract I…” (Internet).  i.e. ensure that it has the green dot next to it.

Your phone should now be able to go back onto 3G.

Step 9: Re-install your Apps

If you now go to Market, it should synchronise and show you your paid apps that you have previously bought on your primary Google Account.

As you have wiped all of the data on the phone (not the SD Card), you’ll either need to re-sync the data (all the Google Stuff), or restore it using Titanium. I’m just progressing through this now.

Other things I had to do

Some final, hopefully useful, things I had to do:

• I used ROM Managers ‘Fix Permissions’. It didn’t do any harm and I thought it would fix the next problem.
• My Google search widget on the home screen failed.  After Googling it, I discovered that it is a fairly common problem after reflashing a device.  Just delete it and put it on again and it should work.

And Finally …

Well that’s it. The recovery image being replaced on Stock 1.6 held me up for a bit and I had to do a load of reading to get it right. So far everything seems good on CM6; it’s seems faster, it is smoother and it has multi-account support.

I also must thank that amazing work of the CyanogenMod team, ROM Manager and all of the people testing these ROMs.  Without them we would be stuck on carrier or manufacturer ROMs with no hope of upgrade.

The following is a list of links that may prove useful to you:

• Full Update Guide – HTC Magic (32B)
• Universal Androot
• [Complete Guide] How To Flash A Custom ROM To Your Android Phone With ROM Manager + Full Backup & Restore
• [Complete Guide] How To Fully Back Up And Restore Your Android Phone Using Nandroid Backup
• ROM Manager
• Can’t Reboot into Recovery on MT3G w/ ClockworkMod 2.5.0.1
• Fastboot (a description)
• Rooting your HTC Magic Android Phone with CyanogenMod
• Titanium Backup
• HTC Magic page at Wikipedia
• CyanogenMod Wiki
• Android at Wikipedia

Well software source control or version control systems.  I tend to use two: subversion and git.  Actually, I tend to use git whenever I can including when I have to use subversion. git svn is, in US terms, awesum.  One debate I have had on and off is whether to:

• Do development on trunk and put releases in branches — OR
• Do development on branches, merge to trunk, and tag releases (or put them in their own branches).

I’ve previously done the former, but I’m rapidly shifting to the latter.  My rather simple reasoning is that I want trunk/master to be, to all intents and purposes, a working copy of the latest software.  i.e. the software that is in trunk/master passes its tests, builds and basically does what it says it should on the tin.  The developing software is done on a branch and only merged back to trunk/master when it passes its tests.  Thus the process is:

1. Branch
2. Develop & test
3. Merge from trunk/master to the branch (i.e. pull any updates)
4. Test, test, test
5. Review code with buddy, senior dev, etc. (depending on your set-up)
6. Merge back into trunk as a finished feature.

Even better, only work on code that is actually described in a ticket, bug, or on something like pivotal.  That way you stay focussed and only spend time on developing features/fixing bugs that the ‘customer’ actually wants.

The only question then is what you do with releases?  My initial guess is to tag them as a release on trunk/master.   Then if you get a bug and have to maintain the old release then to branch at that point and maintain the release as its own ‘master’ release branch.

However, one presentation tried to tackle the immensely challenging science of networking and attempted to explain the academic difference between complex and complicated.  Whilst the explanation of ‘complicated’ was good, ‘complex’ was almost just left as being ‘not like complicated’  – the explanation didn’t really explore the concept due, most likely, to a lack of time.  I’m going to try to touch on it here.

(Update:  This is about complexity theory, rather than the normal English use of the words.  Sorry for any confusion!)

Complicated was described using the Boeing 747 example.  This one often pops up in popular literature as being an example of something that has a huge number of parts that fit together in a staggeringly intricate way.  View a plane as a whole you don’t see the myriad parts of the engine for example.  “Wow, that looks complex!” you might exclaim.  Well, no, it looks complicated.  You could, with sufficient time, completely dissasemble a 747 and lay out all of the constituent part.  You also could understand the purpose and of each and every part and how they fit together.  With time, there would be no mystery why the engine’s fan blade is shaped a particular way.

Complex on the other hand, describes an entirely different set of phenomena.  The Boeing 747 is an example of a complicated system, where system is used to describe a set of components which together achieve a purpose.  The trouble with analysing complex systems is that looking at the parts tells you almost nothing about the system.  That is, you have to look at the interactions and feedback between the parts and the emergent behaviour that the interactions and feedback bring about.

It is probably easier to take a few examples.  Birds flocking and termite colonies are examples complex systems as are organisations and society.

Birds flocking is a bit of a classic and programs have been written to simulate it.  Each agent (or ‘bird’) only has a small set of behavioral characteristics or rules when flocking and completely analysing a bird does not give an inkling of the beauty of the motion that a flock of birds draws in the sky.  The behaviour of the flock emerges from the interactions of the whole and, critically, cannot be predicted by looking separately at the behaviour of each of the birds.

Termites are interesting in that they are quite simple and can be examined from a physical and biological perspective of the individual termite.  However, again, their social functions and mound building can only be studied at the level of the colony the social behaviours only emerge from the non-linear interactions and feedback between the termites which is one of the features of a complex system.

When humans are part of a system, each agent can make their own decisions and they will only do that based on their own partial perspectives of their own understanding of the system.  Hence society and politics.

So complex systems are non linear, involve feedback loops and resist reductive analysis; they can only be analysed at the macro level where the behaviour of the system emerges from it parts.