Geoffrey Meredith
Thoughts on Technology

Blog

(posted on 11 Aug 2009)

A few weeks ago, serendipity provided me an older, but quite serviceable Macbook.  I've never seriously exercised a Mac in my day to day life due somewhat to the steep buy-in cost but mostly because of inertia.  In that time period I've spent quite a bit of time building up Max (the network name for the Macbook) to match my needs in a notebook computer.  In addition, I decided to use it as the host computer for my new iPhone.  I've talked before about my dislike for iTunes and other problems I've had making Max usable in my life but this is only half of the story.

I have two fairly distinct uses for a notebook computer.  The first is similar to most people's use: Firefox gives me a window to the world.  I consume various media using Firefox or iTunes and indirectly via an iPhone.  I create a few documents or spreadsheets via Google Docs or Open Office. 

The second use for a notebook is less typical.  I write software and I need a complex set of tools to allow me to do that development.  While I still do ocational support of very old Windows desktop and server software that I developed in the 90's, I have been developing for Internet platforms almost exclusively for the last 12 years or so.  (With a few side tracks into the mobile world).  That development has been targeted toward Linux centric technologies with a primary focus on PHP, Ruby, and Python using MySQL.  To do that kind of development you need a development computer that supports these technologies.

OS X, with it's roots in BSD, would seem like a great platform for this.  A slick user interface with a familiar UNIXly environment under the skin was pretty enticing.  So with that, I started the process of getting all of these technologies working on Max.  I quickly found out that many of the software packages that I tried to install would not work with Tiger, OS X version 10.4, only one generation old.  So I upgraded to Leopard, the lastest version of OS X and only a month or so before the new Snow Leopard comes out.  (Interesting: there is no free upgrade to Snow Leopard for purchasers of Leopard just a month before shipping.  Microsoft would not get away with this for Windows.)  What also prompted the upgrade was the issues I was getting trying to install MySQL-python connector.

The archive and install upgrade of Leopard went well and I was pleased with the improvements.  But it didn't solve my MySQL-python issue.  It did allow me to install Aptana, my preferred development editor.  After much experimentation and many hours lost, I did get the MySQL-python connector working.  I've long been using PhpMyAdmin for administering MySQL.  It's so much easier than using a command line MySQL client.  That took many hours more.  Should have been 10 minutes.  The only pieces that were easy were the ones that were already installed and often to get other things installed, I needed to install a different version from the default Leopard version.  That leads to many copies of almost identical software scattered over the BSD/OSX Frankenstein file structure.  Each tool that I installed, complicated the situation.

In the end, I've given up trying to force my requirements on Max.  I'm sure that with a couple more weeks of work, I'd get it working the way that I want.  The thought struck me that at that point, I'd have a computer that meets my needs only as well as my 3 year old Dell notebook.  It might be a little faster but I cannot tell.  There is no software that I *need* OS X to run and lots of apps that while not critical, will not run on OS X.  My experience with OS X has not shown me any compelling reason to use it over Windows 7.  I understand that I'm not a tyical user and that there are many for whom OS X works well and might be ideal. 

I think that if I had been coming from OS X and trying out Windows 7, I'd likely have an overall similar experience.  In a lot of ways, OSX, Windows and Linux have become quite close in terms of features but they all differ in the details of how you get things done.  In the end, it comes down to personal preferences and experience as to what works best for an individual.

So my next "experiment" is to see what Max is like running Windows 7.  Will the hardware work better for me than the Dell with the same OS?  If nothing else, I can easily install the software that I need on it in a couple of hours, and it will hold me over until I figure out my next portable computing platform.  I've got Boot Camp and Windows 7 already setup.  That was easy.  Now to duplicate my desktop setup.

(posted on 6 Aug 2009)
Today's DDoS attack on Twitter highlights yet another reason why the Twitter approach to global communications is wrong.  One of the wonderful attributes of the Internet is that it provides a platform that for building distributed applications on.  Commercial operations repeatedly try to build centralized tools on this decentralized system.  These applications work at smaller scales but at a global scale, it is just not going to work well.  It will be brittle and susceptible to a single point of failure problems like DDoS attacks.
(posted on 6 Aug 2009)

I have upgraded my "new" macbook from Tiger to Leopard and it turns out have been a really good idea.  When using Tiger, I had this concept of OS X being really clunky.  I chalked that up to being new to the OS and not real failings in the operating system.

The most noticeable improvement for me is in the Finder's ability to browse remote Windows network drives.  Leopard's "Places" makes network drives much more accessible.  The WiFi menu is not so annoying as in Tiger where it would wait and do a rescan of available networks before it could come up.

A lot of functionality under Leopard just seemed more usable.  I am a bit surprised by this.  I regularly switch between Windows XP, Vista and Windows 7 and while there are differences, the basics just do not seem that different.  I'm sure that part of this is that I've been using Windows for over 20 years so a lot of it is just second nature to me.  I guess I'm not the idea switcher as I've got such depth of knowledge in Windows that using OS X is often frustrating to me.  Luckily Leopard seems less frustrating.  Looking forward to Snow Leopard!

(posted on 30 Jul 2009)

I have fairly actively avoided Apple products most of my life so diving into a Macbook/iPhone setup has been quite an experience.

First of all, I really like the iPhone 3GS.  I was really hoping that the Android platform would become a serious competitor to the iPhone but so far, it really hasn't so I've decided on the iPhone.  I haven't been disappointed by much on the iPhone.  The web browsing experience is fairly fast and usable.  This is a great improvement from Virgin Mobile's poky mobile browser on the LG Rumour that I had been using.

One of my biggest concerns was the need to use iTunes.  Past experiences with it have not been positive to the point that we've joked that the Windows version should be caught by anti-virus programs as a virus.  Now that I've got a Mac, I figured that iTunes should be a well integrated and highly functional part of the environment.  Sadly, iTunes seems to be as poorly written on OS X as it is on Windows.  I have not had so many problems with a mature piece of software from a major developer in my 30 years using and developing software for computers. 

My first adventure with iTunes was getting a library of 7000 songs onto the iPhone.  That would seem easy to do.  Import the files across the network from their repository on a Windows server.  I do this every time I install a media player, usually WinAmp, on any other computer on the network.  It generally tasks under an hour to go through the files, get the ID3 tags and build a database of songs.  So I started the process with the iTunes "Add to library" function.  It would have been nice to just enter the UNC path to the song archive but no, I had to use the finder to map a "disk" to the that location on the network and then access my songs from there.  Lots of mousing around where a few keystrokes should have done it. 

So the import began.  After a couple of hours, I realized that this was not going to be a quick process.  Not only that but the computer was completely unusable while iTunes with in this import process.  So I left it until the next morning.  Well it was "done" but it had only imported about half of the songs.  It looked like the network connection to the share where the files were located had "gone away".  The machine was on a wired connection to a rock solid network and server.  Not sure what went wrong but I started the process again.  Many hours went by and this time it looked like iTunes could see all of the songs.  They played well.

I was ready to start putting music on my new iPhone.  It was about 3 days old at that point and I had not been able to get any music on it.  So I started to sync with the iPhone.  I left it for awhile.  When I came back only about 500 songs had sync'd.  Why only those songs?  They all seemed to be songs that had poor ID3 information in them.  Maybe iTunes hadn't imported the songs well so I remove all the songs and started importing again.  Another night went by.  Next day, the iPhone sync seemed to import the same 500 songs only.

One thing that stuck me was that maybe iTunes didn't like it that I had imported songs but had told it to leave those songs on the network and not copy them to the Macbook.  So, remove all the songs out of iTunes, and start yet another import with copying of the songs.  Before I did that, I imported a small subsection of the music library and it seemed to get onto the iPhone without any problems.  The big import took about 2 days but when it was done, I was finally able to get my music on the iPhone.  Why this would not work well over the network, I have no idea.

Next task: figure what the "Apple Way" is with respect to podcasts.  Arg!

(posted on 28 Jul 2009)

I haven't used an Apple product seriously in a long time.  In the early 90's I worked on a project where I had to use a Mac for a few weeks but nothing since then other than occasionally borrowing a friend's Mac to check email, etc.

A couple of weeks ago I acquired an 18 month old Macbook with a dead hard drive.  When I got it you could hear the heads tearing themselves apart and then nothing.  A nice 320GB 7200 rpm drive fixed that up.  I decided to use this Macbook as my portable computer.  Replacing a 3 year old Dell.  The Dell still works great, especially with Windows 7 running on it but I thought that the Mac experience would be good to have.

My desktop environment is still a mix of various versions of Windows and Linux machines.  That makes six boxes under my desk.  It is also why I write this sitting outside, under a tree.  It just too hot in my office with all that equipment.

About a week after I got the Mac running, Rochelle and I decided to take the plunge and go for iPhones.  So I've a few steps into the Apple world.

It will be interesting to see if there are any surprises in this experiment.  I have been writing software in one form or another for over 30 years.  For many years DOS/Windows and UNIX/Linux have been the software development platforms of choice but over the last couple of years there has developed a large continent of Internet software developers who prefer Macs to do their development.  Good graphics and video manipulation tools have obviously had a lot to do with that but more hard core developers have been promoting the use of Macs.  It was particularly noticeable when I started doing some Ruby on Rails development a couple of years ago.  Windows was definitely not the ideal platform for Rails development.  I had been contemplating purchasing a Mac when this one fell in my lap.

I'm not going to go cold turkey into a Mac only environment.  I will still do most of my work on a Windows 7 based desktop machine but this new Mac will allow me to explore the possibilities.  I hope to replicate a basic software development environment on this machine for my mix of PHP, Rails, Python projects that I have either under active development or maintenance.

(posted on 27 Mar 2009)

This TechCrunch story talks about how it's hard to create a music streaming startup.  The problem here is that these startups are trying to bring the old radio business model onto the internet.  That model has the radio stations paying the music industry for promoting their product.  To pay for privilege, the radio industry has plastered there service with ads.  Online, where ad rates are much lower but the streaming rates are much higher, the economics just don't make sense.

Why should the old analog model that was based on the scarcity of government owned radio frequency bandwidth make any sense in the long tail enviornment of the internet where the only scarcity is attention.

So how should the music industry work on the internet?

The music industry should be paying startups to help them promote their artists.  This should be under the control of both the artists and the listeners.  Listeners should no longer be force feed what the music industry thinks that they should hear.  Alternatively, artists and labels should be able to be able to promote themselves as they feel the market might find interesting with a minimum of disintermediation.  Where those two spheres intersect, we get maximum value for listeners and ideal promotional opportunities for the music industry.

(posted on 13 Jul 2008)

I posted a job ad on Craigslist.org yesterday that got quickly flagged and removed.  You don't get any indication as to why and ad was removed, just a link to a forum where you can post the details of your ad and get suggestions as to what you did wrong.  I had no idea as to why my ad was pulled so I spent some time on that forum looking for clues.  Nothing in the examples I saw there helped me understand what was wrong with my ad so I posted my ad to the forum and and awaited responses.  I only received one comment about my ad.  The comment was sarcastic and suggested that the compensation was only appropriate for a third world country.  Maybe I'm cheap or out of touch with salary expectations (or both) but I do think that there would have been people interested in responding to the ad.

The responses to a number of other rejected ads seem to expose either personal or political agenda.  People just didn't like the ads because they didn't properly address their political ideals, even if those political ideas were tangential to the posting.  It became obvious that to post in any particular category and city, you have to abide by mostly unwritten "community standards".  Who's standards are those "community standards"?  It's not the community of people in that city interested in that category, it's the much smaller community that comb through Craigslist ads looking for ones that do live up to their standards.  That to me sounds like a vigilante mob; the darker side of crowdsourcing. 

I don't think that it has to be this way.  I think there are technological solutions to improving moderation on websites like Craigslist.  Meta moderation and sophisticated algorithms that websites like Slashdot.org, Digg.com, or even the use of more descriptive flagging, can be added to raise moderation beyond the level of vigilantism.

Of course, this may be the way that Craigslist wants their website to behave.  It's certainly within their rights to do so.  I actually see an opportunity here for someone to build a better Craigslist.

(posted on 26 Jun 2008)
Among the many things that I look after, I manage the email for several hundred domain names.  A large portion of these domains are for individual artist websites and thus have only a couple of actual email addresses.  In most cases I just forward any inbound email to the artist's ISP or web mail email account.  We don't filter that email in any way so everything, including spam, gets forwarded to the owners actually email address.  We have avoided spam filtering because most people already have spam filtering on their email account so our filtering would not be beneficial for the recipient.  I'm sure that the spam filtering efforts by GMail, Comcast, Yahoo, etc is much better that I'm going to be able to implement.

Over the last month or so we've started running into issues with Comcast and AT&T blocking all email from our servers due to the fact that they receive what they consider spam from our servers.  We have gotten our servers unblocked but today, Comcast has blocked us again.  So, to be able to deliver email to Comcast we have to "clean" all email that passes through our servers.  We have no idea about what the triggers are for Comcast to block a server.  The barrier is likely to be fairly low as we don't have all that much email traffic in total.  So to keep our standing with Comcast, we will have to be brutal.  We will have to consider any email that might possibly be spam as spam and bounce it.  If only a tiny percentage of spam gets though our filters, we might get blocked again.  The net effect is that some legitimate email will bounced.

While neither us nor our customers are doing anything wrong, Comcast is forcing us to not just tag potential spam as spam but forcing us to block it entirely.  Essentially they are pushing their problems on us.

The net effect of all this is that Comcast will be forcing many smaller operations that process smaller amounts of email to find their own solutions to deal with the "Comcast" email problem.  Each operator will find a way that will cost in aggregate thousands, maybe millions of man-hours of effort and will at a net, reduce the percentage of successful legitimate email deliveries.  Spam has made email less useful but these efforts by Comcast will be adding some of the last few nails to the email coffin.  I'd love to see email disappear but it won't until something better takes it's place.
My previous post about the Internet Operating System compared the Internet's structure and operation to a stand-alone computer.  In this post I want to take that another step further.

I have the feeling that the state of the Internet now is much like stand-alone computers were just before the introduction of the IBM PC in 1981.  A lot of the pieces of the PC revolution were there, but no one had quite put them all together.  What the PC did was put control of serious computing resources into the hands of individuals.  We are now waiting for the Internet analog to this revolution.  I think that will happen when people control their data own on the internet.

That data control is not just in the "Data Portability" vision of being able to copy data from one walled garden to the next, but in the ability to store your data in a single datastore of your choosing and that you control completely. You can then allow selective access to to your data by external services that you want to use.  I think that Amazon.com's S3 is the start of the kind of service where you could store data.  Not that S3 has the complete functionality required to support this model but it could be based on top of S3.  Having your own datastore is like being in control of the hard drive on your computer.  You load applications and tell those applications what data to work with.  In that same way, you could allow a web based service such as Adobe Photoshop Express, to access some photos in your datastore, do some online processing and after it's done, store the results back to your datastore.  You can already do this with your photos stored on Flickr and a couple of other photo sites.  Adobe's got the right idea but there is no open protocol for that would allow them to reach the photos on my own personal server.

In a similar vein, we have Facebook, Google, Yahoo, Microsoft and many smaller players fighting over control of "the social graph".  The "right" way to handle this is to allow me to store and control my part of the social graph and then selectively allow other services to have access to that.  There would no longer be a need to give some new tool your account credentials to your GMail, Facebook, and other services.  Just point them at your datastore and tell your datastore what personal data that the service can have.

This model really is the holy grail social computing from a user's perspective.  It's deadly to a social aggregator's perspective (such as Facebook) as there isn't much left for them once they the user gets rescued from their lock-in.  I also see this as a significant component of the next version of the Internet Operating System.
For many years, there has been talk about the coming "Internet OS".  Given Google's current dominance on the internet, may feel that it will come from them and often call it "The Google OS" (not to be confused with Goobuntu. Google's internal Ubuntu based OS).  The thing is, the Internet OS is already here and has been for roughly 30 years.  TCP/IP is fundamentally the internet operating system.

In the stand-alone computer world, an operating system is the software that manages and ties together the hardware components, usually via hardware drivers.  The operating system exposes APIs that allow application programs to interact with the hardware as well as the lifecycle of the applications themselves. Over time, computer operating systems have grown by adding layers over this fundamental "kernel".  These new APIs provide rich, event driven presentation layers such as Windows and Mac OS.  They also provide such things as file systems, security, etc.

Since the 70's, TCP/IP has been the operating system kernel that has tied together the various bits and pieces of the internet, (computers, switches, routers, etc.).  This has allowed for a variety protocols to be developed that are roughly the equivalent of an operating system API.  In fact, from a software point of view, most programs use the protocols via an API that implements the protocol.  Once the TCP/IP kernel was created, a set of protocols developed on top of it.  These are the APIs of the Internet OS.  Those protocols started out with basic file transfers but soon added SMTP email, remote login, and more recently, the HTTP web, VoIP and to some extent, instant messaging.  [There are lots of other protocols in there but most haven't become mainstream or, as in the example of DNS, we don't often think of them until they break.]

An operation system is not all that useful if it's just a bunch of APIs.  You have utility programs and shells to allow people to interact with it and you have major applications such as word processors and spreadsheet software.  In the same way, we have SMTP protocol clients such as Outlook and Thunderbird, HTTP clients such as Internet Explorer, Firefox, Safari and Opera, etc.  We also have lots of utilities such as ping, traceroute, etc that provide and direct interaction with TCP/IP.

So what we are living with now is Version X (no one has been keeping count) of the Internet.  What people are eagerly awaiting is development of new protocols that will allow us to get beyond the "old" models of interaction on the internet that involve email and web pages and basic media viewing.  We've gone a long way in creating great interaction models with these basic protocols.  Web 2.0 has started to give us insight into what the future possibilities are but we need to take these ideas and encode them into internet scale protocols.

I've follow up this post with some ideas about what I see as the emerging, next generation technologies and protocols that will make up the Internet OS Version X+1.
older blog items...