I was recently on holiday up in Vancouver, BC. Unfortunately my GPS unit had not been preloaded with Canada maps, so in the interest of not getting lost, I elected to open a map once or twice on my smartphone. I knew that this would be charged as “roaming” data, and as a smart person I had done my research, and established that AT&T would be charging me the exorbitant price of… $0.015 per kB. Didn’t seem soooo terrible – I figured that opening a map would use a handful of kB tops. If I was unlucky I was looking at a few dollars worth of usage. How wrong I was.

Total usage: 4.6MB. Total charge: $69. FML.

Let’s put that in perspective. I pay $33 per month for internet at home. I can use up to 250GB on my plan without repercussion, although let’s say I average around 100GB. That means I pay around $0.000000315 per kB.

The upshot of this, is that the data I used in Canada cost me more than 47,000 times as much per byte than my home internet plan. Forty. Seven. Thousand. For those keeping score, that means that if the roaming charge was the standard rate, I would be getting a 99.9979% discount on my internet bill every month. You can’t make these numbers up, we’re talking mind-bogglingly disparate numbers here.

Now don’t get me wrong, I understand the implications of roaming. AT&T has to negotiate with third-party providers to use their networks and their bandwidth when I’m out of the country. I understand that there is a significant difference between the cost of transferring data over a cable connection and a 3G connection. But if you think that all of those issues justify a 4.7 million percent markup, you have got to be pulling my leg. Canada is a first-world country, with (presumably) first-world cell networks and internet access. AT&T can give me an unlimited smartphone data plan for $20/mth, but can’t negotiate a better rate than $60 for 4MB of data?!

Give me a fucking break.

SELECT /*!40001 SQL_NO_CACHE */ * FROM `phpbb_search_wordmatch`

Very mysterious. The offending table is one relating to my phpBB instance. I did not have any further information at hand to work out what was going on, and so my first port of call in attempting to resolve the issue was to adjust my phpBB search settings. However, a few days later, the error occurred again – back to the drawing board.

My next step was to inspect the source code of phpBB. However, I could not find anything that would produce SQL like the above statement.

And then it occurred to me. I started receiving the error messages about the same time as I set up a Cron database backup job. Sure enough, all the errors were coming through at around the same time of day, coinciding with when the backups were scheduled to run. A quick web search confirmed that running mysqldump against very large tables was a typical culprit for long-running SQL statements.

Once I knew the source of the error, the fix was very simple: Because the table in question is just an index table for the purposes of searching posts (i.e. not essential data to backup), it was just a case of adding –ignore-table=db.phpbb_search_wordmatch to the mysqldump command line in my cron job. Now, the search index table is skipped in my backups, resulting in smaller, faster backups, and no more error messages!

I couldn’t find much information about this problem specific to phpBB or Webfaction, so hopefully this helps someone else out!

If you have been browsing the internet lately, you have more than likely seen a Facebook Connect box. It looks like this:

This particular screenshot is taken from Gameplanet Forums, but Facebook Connect makes it easy for any website developer to embed the panel into their website. I could put one on tom.net.nz, if I was so inclined.

Now, Facebook would argue that this frame is a naive, harmless feature, because the information is never passed directly to the website in question, but rather the website just embeds a little piece of code, and Facebook generates the actual content of the pane. The code embedded is identical for any user visiting the website.

This is all well and good, however (!), the content that Facebook generates for this pane will differ depending on whether or not the user viewing it is currently logged into Facebook. If they are, then Facebook tries to show information more “relevant” to that user. For example, in the above screenshot, “Matt” is my friend on Facebook (and the only person in my friends who has “liked” Gameplanet on Facebook). The other people are generated randomly, but no matter how many times I refresh the page, Matt will always appear in the list. Do you see where this is going?

One might assume that Facebook has put some measures in place to stop the site from scraping this information, however the tech savvy can follow the following link which generates the box for Gameplanet’s Facebook Connect pane: link. If you view the source, there are all the names, in plain HTML, with links to both the photo, and (perhaps more disturbingly), the profile of each person. I was also able to scrape my own user ID from the HTML. Fortunately, in most modern browsers, XSS (Cross-Site Scripting) protection prevents the parent page from accessing the DOM of Facebook’s frame, but this is still a major potential security problem for older browsers which don’t have such protection built-in. By inspecting the list of users on a few page loads and looking for duplicate names, a malicious site could ascertain who is friends with the user browsing the page. The parameters passed to the frame source allow significant customization of the response, for example with a bit of tweaking I was able to come up with the following source, which now shows 100 users instead of the default 15. To take it a step further, a malicious page could potentially load this frame without even showing it, meaning the user would be completely unaware that the site was doing anything to do with Facebook, meanwhile it’s scraping the user’s private information.

Of course, those using a remotely modern browser do not have to worry about this sort of attack… But I think it does highlight the potential risks associated with these completely unnecessary “features” – not to mention the dangers of using an out-of-date browser. I would have hoped that at the very least Facebook would have performed some source obfuscation or dynamic JavaScript DOM population.

That’s more than 4 years ago now, and it’s kind of saddening to see how Facebook has changed in that time. When I first joined, I immediately liked it for the following reasons:

• It did photos better than any other site I’d seen, with photo tagging and a simple interface (I was on MySpace, Friendster and Bebo at the time, none of which had photo tagging).
• You couldn’t do any customisation of page appearance. Everyone’s profile was clean and uncluttered. (BIG contrast with both MySpace and Bebo). There wasn’t any extra cruft, and you couldn’t add images and music and videos and so on into your profile. This alone made me love Facebook more than anything I’d used before. My eyes would no longer bleed when looking at peoples’ profiles. No more sparkly stuff, or god awful music that started playing as soon as you opened a page. Thank God.
• Privacy was the default, not a hidden option. You couldn’t see anyone’s stuff unless they opted to make it public, or you added them as a friend.

Overall, it just felt much nicer and cleaner than anything I’d used before. I promptly told all my university-attending friends about it, and a few months later, ditched the other three sites. Then started the slow and steady decline. You may or may not have been around to see some of these changes:

• The opening of registration to anyone and everyone. Suddenly the site was full of preteens and teenagers that (quite frankly) I was happy to have left behind on MySpace and Bebo. When it was only open to university students, it had an “exclusive” feel to it, which was nice.
• The introduction of applications. This one particularly pissed me off. I was getting constant spam from other people “inviting” me to use the applications they were using, and my news feed was full of junk about other peoples’ application activity. Worst of all, the thing that originally had me most excited (the clean, uncluttered appearance of profiles) was now pretty severely compromised. While you still couldn’t change the appearance of your page in terms of background colours/images, and while the layout was still roughly the same, pages were now full of “fishtanks” and “DO U LIEK ME?” questionnaires, and other such junk. This is now better under control, with applications appearing either in the small column to the left of profiles, or in the “Boxes” tab, but it was certainly annoying at the time.
• The gradual erosion of user privacy over the years. This has almost happened so slowly that it’s been unnoticeable, but Facebook is (at its current stage of development) trying to stick its finger in too many pies. It is pandering to its partners and developers, by giving them more access to user information, at the cost of user privacy. With the likes of Beacon (now shut down, two years after its inception), Connections and Open Graph now making your public information available to third parties, it has now reached the point where it seems they no longer care about their users as anything other than a revenue source. The information being shared with third parties notably includes information that you can set the “Visibility” of to “Only Friends” in your privacy settings. Apparently Visibility only refers to other Facebook users, and some third parties are exempt from such restrictions. Users essentially have no control at all over where this information could end up.
• Along with the occasional reduction in default privacy settings (or removal of privacy completely from certain items), Facebook seems to be deliberately making it difficult for you to opt out of these settings.

Perhaps even more worrying, is how few people are actually even aware of this erosion of privacy. Facebook have done a good job of keeping it pretty well hidden, or glazed over as “enhancing the experience”. Of course, most people (myself included) don’t read privacy policies (often pages and pages worth) every few months.

Of course, a social networking site is only useful if there are a good proportion of the people you care about using it. With that in mind, Facebook is still good from the perspective of sharing photos, organising events, and communicating with friends. However, I have stripped most of my personal information from the site (bar photos), and will gladly move to a new platform if it can grab me in the same way that Facebook did when it first appeared on the scene.

In particular, I’ll be keeping an eye on Diaspora, an intriguing project about to start development, which promises to deliver on the concept of an “open source” social network. In a nutshell, it’s a network of personal “nodes”. Each person on the network owns their own node, to which they can add whatever information they like, and access it from anywhere. In turn, they have fine-grained control over who can see that information. Think about it like your own personal house that contains your personal information (basically everything that you’d otherwise be sharing on Facebook/Flickr/LastFM/Twitter/etc), and you can open the door two whomever you (and only you) choose. Because you own the house, you can demolish it at any time, or add and remove furniture as you please. It’s also being developed by a bunch of super nerds, so I can totally get behind it:

Hopefully it amounts to something! My biggest concern is that it’s very technical and geeky at this point in time, so they will need a good marketing team with a pitch for the masses before it gains any real ground.

• Home
• Pools

Hopefully if I keep this up, I’ll be inspired to write some actual code…

And now… Now I’m not sure if I’ll ever be bothered to do anything with it. If I haven’t started coding come the time when the domain expires, I might just let it go. I just didn’t get inspired enough to start implementation. I think I need a really fresh, engaging project to start on. Open to ideas.

Recently I’ve moved into a new flat. Our ISP at our old flat was TelstraClear, and we rang them the week before moving to arrange transferring our connection to the new location. We were only moving a couple of blocks, and everything was arranged for the connection to go through without a hitch. Cue moving in, where we quickly discover that there is an existing connection at the house that is still connected from the previous tenants. A bill in the mail reveals that the previous tenants were also on TelstraClear. We weren’t scheduled to have our line connected for a week from the date that we moved in, but because the previous tenants were connected using the same ISP, our modem was able to connect and authenticate fine, meaning (in theory) that we could use the internet over the old line, while we waited for our proper connection to be put in.

Ever the cautious one, my flatmate suggested that we should double-check it would be ok, as he had a feeling we might end up paying some sort of fee if we used our account over someone else’s line. And so I decided to ring TelstraClear customer service to find out what the deal was. After waiting on hold for literally an hour (I wish I was exaggerating), I was informed that no, in fact, we could not use our internet account over someone else’s line, as they would be charged for the internet usage. This seemed odd to me, but I took their word for it. Long story short, we didn’t have internet for the first week of being at the flat.

So the day rolled around that we were supposed to be connected with our proper line… And, nothing much happened. The old line was still connected. (This was established by calling out to a cellphone from the line, and checking the caller ID). Cue another 30 minute wait for TelstraClear customer service. After explaining the situation in full to the technician and waiting around on hold some more, it eventuates that the old line was unable to be disconnected on the scheduled date due to a hold being placed on the line. Apparently the previous tenants had rung up and done something because they were moving out (which was apparently “the right thing for them to do”), and TelstraClear was waiting to hear back from them about something (which they were never going to hear back about, because the tenants had MOVED OUT, hello?! how does this make sense at all?!), and this waiting period resulted in a hold being placed on the line. Fortunately, that hold expired a few days later, meaning we could finally be connected on our proper line and start using the internet, 3-4 days after the expected switchover time. Not too terrible. In passing, I decided to pose the same question I had waited an hour on hold the previous week to ask: Can we use our internet account over someone else’s line without incurring fees or charging the other person? Imagine my surprise when the technician gave me the opposite advice – that we could, in fact, use our internet login without any issues whatsoever, and it would appear on our statement as normal. So in a nutshell, I waited an hour on hold the week before, to be told something that was incorrect, and ultimately meant that we were without internet for a week when we could have been using it the whole time. Got to be shitting me.

I wish the story ended there, but no.

3 days later, the old line was disconnected as expected in the morning, and the line (presumably the correct one) came up again later that afternoon. We were good to go. I joyously connected and caught up with my downloading. All seemed right in the internet world. But then, DRAMA.

About a week later, I was curious as to what our new number was (as we had been told we would be getting a different one to the one at our previous address). I rang my cellphone from the landline, and what do you know? The number that came up on the caller ID was the same as the number on the line from the previous tenants. So, in a nutshell, they had disconnected the old line, then reconnected the same old line instead of setting up our proper line. Got. To. Be. Shitting. Me.

Cue more waiting on hold to TelstraClear customer service. An aside about their customer service… When you finally get connected to where you need to be, the technicians are helpful and friendly, however, everything else about it sucks. When first ringing up, you get a computerised voice that “understands what you say”, and a menu of options. However, even after I had learnt which department I needed (thanks to having to make so many phone calls, and being told repeatedly “let me transfer you to X”), THE DEPARTMENT ISN’T IN THE LIST OF OPTIONS! Meaning that I would have to pick an option sounding close to what I wanted (in the end I was pretty much picking one at random), hope like hell that it didn’t have a long wait time, then ask to be transferred to the actual department that I needed. The department I needed invariably had a 30+ minute queue. On my first call, I made a poor choice at the first menu, meaning I was waiting on hold for 30 minutes to the wrong department, and when I finally got through, I had a 30 second conversation, and was promptly transferred to the end of another 30 minute queue to the actual department. Total wait time: 1 hour. Surely if you wait for 30 minutes to talk to one department, they could have the decency to bump you up a bit for the next queue. I eventually learnt that their “sign up for a new account” department was well staffed and never had any queues (surprise, surprise), so I would just pick that option and pretend I did it on accident, and get them to transfer me to the actual department I wanted.

Anyway, where was I? Ahh yes, someone’s failure (presumably TelstraClear’s, although they do outsource the connections to another company called Chorus) resulting in the old line being reconnected. Long story short, they needed to re-disconnect the old line, and finally connect our line. So, on Monday (3 days ago at the time of writing), the line (and our internet along with it) was once again disconnected as expected. I don’t really understand why they couldn’t have connected up our line at the time they disconnected the old one, but here we are on Thursday, still with no connection, and with no idea (even after two more excruciating calls to customer service) when we will be reconnected. So, in a nutshell, due to no fault of our own, we are once again without internet or a phone for an indeterminate period of time.

I’m not happy.

In other miscellaneous news, summer is wrapping up, which means that in about 3 months the ski season will be starting again – can’t wait! This year a few of us are planning a week-long South Island trip, which should be totally epic!

Exactly what they sound like

• Awkward Boners
• Awkward Family Photos
• F My Life
• Oddly Specific
• Texts From Last Night
• Unhappy Hipsters

People I like

• Brian King
• David Thorne
• Patrick Melton
• Remi Gaillard
• The Oatmeal

Wow, that list is sort of pitiful. More to be added when I remember them?

If anything, the music industry should have more respect for file-sharers, as they are their most valuable consumers. They are ahead of the curve and actually leading the way for the future of digital music, buying more digital music than anyone else. It’s the music industry that has to change, not the other way around.

TorrentFreak outlines how, once again, the music industry is overstating the piracy “threat”.