Mac@Work : The Ultimate Business Machine

I've updated both the Lion and Snow Leopard versions of DNS Agent, my app for dynamically updating dyndns.com's servers with your Mac's current public IP address.

In April last year, I updated DNS Agent for Snow Leopard to 1.0.1 because I realised that dyndns.com servers have started to ignore pings, by working in stealth mode. I ping the dyndns servers to make sure that they were available before attempting to make my updates. So, since pings don't work, I check instead that the DNS Server is actually listening on the appropriate port (for updates from clients) before trying to update to server.

Then sometime recently, dyndns.com servers have also started to ignore telnet probes on these well-known ports. So since these checks don't work anymore, DNS Agent gets stuck waiting to update the dyndns.com server.

So, with version 1.0.3 of DNS Agent for Snow Leopard (and version 2.0.1 of DNS Agent for Lion), I now stop doing these checks and simply proceed to update the dyndns.com server, trusting that it would be there, as they should be.

This should probably (reasonably) work almost all the time. I log any failure to update the dyndns.com server, but up to now, my log shows that updates do work all the time.

Liya for iPad

I'm now working on Luca, the accounting system. I'm doing three versions. Two for Lion, one of which will, hopefully, go to the Mac App Store and have support for iCloud (and conform to Apple's Sandboxing rules) but it'll only use the built-in SQLite database.. The other, we'll sell on our own site and will have the usual support for MySQL and PostgreSQL.

Then there's a third version - Luca for the iPad. It's not quite so easy to port an app from the Mac to the iPad, especially one that exploits the ability to launch multiple windows to offer the user multiple views of the same data. The single window on the iPad does initially feel like a straightjacket. I'm still not sure how this could be done.

P.S.: After this, I have to go back to the server apps. The API calls that I've been using to do most of the magic are going away. What if we can't install our own apps on our Macs beside those Apple-sanctioned ones from the App Store? That'll be the time to find another platform to pour the heart into.

Steve Jobs

I've finished reading "Steve Job". I took my time over it cos I didn't want to come to the end - that had felt sadly like literally closing the book on Jobs.

About half-way thru, I realized I could write this book myself. I've read all the books on Jobs and Apple over the years. Except for Jeff Young's The Journey is the Reward, most, like Jim Carlton's & Mike Malone's were "hatchet jobs".

So, this biography, while being thankfully sympathetic, mostly rehashed the things we already know & thereby disappoints. I kept thinking, you idiot, you get to ask Steve Jobs questions? Jobs, for one, would know the loneliness of the contrarian thinker. You can plumb the depths of his soul.

Liya on the App Stores

I just wanted to feel what the Mac App Store submission process is like. It's slightly simpler than the App Store for iOS apps. Will think about submitting Luca next. Unfortunately, my server apps will never make it to the App Store because of Apple's restrictions on apps that need to gain supervisor/root privileges.

Liya for iPhone has also been updated to 1.1 on the App Store. I had a sudden spike in downloads the last two days. It had previously been anaemic. I wonder why.

Handling NULLs in SQL and NSDate Fields

I've finally managed to solve a long-standing design problem, probably because I'm settled now with Lion and have more time to think.

The problem I was having is with NULLs in date fields. Why do we have NULLs? When we don't know what an actual date will be when we're filling up a form earlier in time - e.g., an insurance claims data entry screen, when we don't know when a surveyor will be able to inspect a vehicle damaged in an accident.

Both PostgreSQL and MySQL will accept NULLs in date fields. But the problem comes when we try to show these dates in a Cocoa Mac OS X (or iOS) screen, using fields that have been formatted as dates. This is because NSDateFormatter requires the data entry to be a properly formatted date value, and NULLs are not.

The problem is compounded when we're working with tabular data and, to retain our sanity, we want all data in a single column to belong to the same data type, so mixing NULLs and proper date values won't do.

I've since worked out a scheme whereby NULL dates from the database get converted to [NSDate distantDate], and when I see as [NSDate distantDate]'s coming back to the database, I convert them to NULLs.

So, inside Cocoa (and iOS), my tabular data will work consistently. Each column is a distinct, consistent data type.

But the problem with showing [NSDate distantDate]'s as actual data values in the user interface is that it clutters up the data in places where the user would have expected blanks in the dates, like below:

[NSDate distantDate] should be 4001-01-01 00:00:00 +0000, but it is 4000-12-31 19:00:00 -0500 in the screen shot above because that is [NSDate distantDate] in Montreal. I set my location to Montreal, or it could be any other city, to make sure my Time Stamp fields continue to work correctly whichever country or locale my Mac happens to be in. This is something I've spent a lot of care on, that my database frameworks and apps like Liya work as fluidly with shifting locales, as Apple's Macs, iPhones and iPads do.

So, I've always wanted NULL dates to show up more naturally, like in the screen shot below, and I've finally solved the problem by writing a custom sub-class for NSDateFormatter. Turns out to be not too hard. I wonder now why I took so long. Probably because it's hard to see clearly when so many things are moving all around, with the migration to Lion.

Somehow I just couldn't make a version work for Snow Leopard anymore - Xcode complaining about some problem with dynamic libraries from 10.6. So, going forward, I'm now going to work only with this Lion version and drop the Snow Leopard version.

I've also submitted this version to the Mac App Store, just to see what the submission process is like. It seems a bit easier than with iOS apps. This is my first Mac App Store app and it's free.

Transactions and Locks in SQL

I'm done comparing the SQL calls I need to do across MySQL and PostgreSQL in order to support transaction processing (i.e., Locks, Begin, Commit or Rollback) in a multi-user version of Luca that I'm working on.

I'm also going to do versions of Luca and Liya for the iPad. I've got a new iPad 2 that I'm looking forward to working on.

Fortunately, the SQL calls for transaction processing are consistent across MySQL and PostgreSQL. They're the same - BEGIN, COMMIT or ROLLBACK - for each platform.

There is a more Oracle-like START TRANSACTION in place of BEGIN in MySQL but BEGIN will do. Why add complexity? Always simplify.

While PostgreSQL has always supported transactions for as long as I can remember, MySQL with MyISAM tables do not. We'll need Innodb, which is the default setting for newly created tables in MySQL 5.5.

Once we can get our Lion installer for MySQL 5.5 done, Hai Hwee has a functionality built into her installer such that it'll migrate forward all the data in your current databases to the new version of MySQL that you are installing. I'm hoping that when that kicks in, all the data in the MySQL tables will then automatically support transaction processing, and then we'll be able to do BEGIN, COMMIT or ROLLBACK from applications like Luca, knowing that we'll be accessing databases that support it.

We also need advisory locks - putting a lock on database updates among consenting apps (I almost said adults). This is a simple form of record locking, to prevent the database from being written to the same place at the same time by different people, using Luca on different client machines. I don't think Luca needs the more sophisticated form of record-locking at the moment, so advisory locks will do for now.

On MySQL it's very easy. SELECT GET_LOCK('Lock1', 10); To test for the lock, it is SELECT IS_FREE_LOCK('lock1'); To release the lock, just do SELECT RELEASE_LOCK('lock1'); I just tested it, accessing MySQL from two client instances. It works great.

On Postgres, the equivalents are SELECT pg_advisory_lock(5), SELECT pg_try_advisory_lock(5), and SELECT pg_advisory_unlock(5).

So two differences - one is the name. The other is that MySQL uses a string for the semaphore while Postgres uses a big integer.

I can hide these differences in my database frameworks so that an app like Luca calls the same code, no matter which type of database it is accessing. So we're good to go.

When I had the idea for Liya and my database access frameworks, I never expected to be able to go so far creating a unifying API that will work for both MySQL and PostgreSQL - mainly because there are so many differences between the two dialects. It's been a long slog. But I'll happily take what I can get.

But just why are we mucking around with this MySQL 5.5 and Lion stuff?

That I think is a good question because that's the nature of things when we're doing development on an Apple platform - it's the "let's get going and take no prisoners" way of thinking.

If the only way I can do these custom builts of MySQL is on Snow Leopard, then I need to keep one of my machines running Snow Leopard. If I can get this problem solved, then I can envisage that every one of my Macs (including the live server) will be running Lion within a month or two. Then I've simplified things. I've got only one platform to think of, going forward. We mustn't forget - we usually only have a two-year window to build on top of all this stability, to get all the new features that we've been dreaming of done, before Apple, in their eternal wisdom, starts to change things all over again.

But why does Apple get away with this? I think it's because the result to the consumer is an endlessly fresh, exciting computing/media tool. But what it costs the developer is this never-ending race to keep up.

Otherwise we'll be like Adobe - trying to keep time in a bottle. Now who wants to be like Adobe. Except for Photoshop, I'm almost all Adobe-free.

Now MySQL 5.5? On 5.5, Innodb replaces MyISAM as the default database engine on MySQL. With Innodb comes support for transaction processing, and commits and rollbacks - things we need when we're working on more conceptually demanding database systems.

Of course, we can already use Innodb even now on MySQL 5.1 and it was so for a long time. But with this as the default, it's easier to just move on and build on it.

I'll hold up on MySQL for the moment and see how we can match up the similar features on PostgreSQL. The way I've been building our database frameworks is that I want to write one piece of code for the customer-facing app, like Luca and Liya, on the Mac, iPhone and iPad, and the database frameworks will take care of the platform differences when they access the relevant databases.

And I'd want to get back to my server apps, like MailServe and DNS Enabler and WebMon. And I need to take a look at Lion server, too. So, lots of work ahead.

More on MySQL 5.5 on Lion

Okay, I'm sure by now that we can't build MySQL 5.5.15 from source on OS X Lion. I'll wait for 5.5.16 to be released and try again.

Am I the only one trying to do this? Nothing else turns up on Google. You can build MySQL 5.5.15 from source on Snow Leopard (32/64 bit fat binary) and that will run on Lion but what i was going for was getting everything built and run on Lion.

I also tried building MySQL Connector/C 6.0.2 on Lion (which is needed to build our database access frameworks at the client ends, like on Mac, iPhone and iPad). That will build OK and I can get a fat 32/64 bit Intel binary. But when I run it, I get lots of NSAutoReleasePools errors. Seems like something to do with threads, and if it were so, then that's like the problem we're having building the whole MySQL 5.5 server on Lion (which failed at doing something like pthreads).

Anyway, I only have a vague understanding of what I'm doing here, at least right now. Will need to do more digging. But I'm sure we'll sort this out. They always get sorted out. Eventually. I'm the rational optimist.

MySQL 5.5 on Lion

I've been working on this the whole weekend but I still can't build MySQL 5.5 from source on Lion. I keep getting errors with the pthread and, after I've found a way around that, there were errors with my_atomics or something like that.

Will leave this for now. Seems to be fixed in the coming 5.5.16 version of MySQL. We'll see then.

Luca on Lion - A Descent into the Walled Garden

I was having problems even opening the Luca project's nib (user interface) files on Xcode 4.1 on Lion. Xcode, moving forward on Lion, has dropped support for custom Interface Builder palette objects. But that was one of the most powerful ideas from NeXT's Objective-C Project Builder/Interface Builder environment - that developers can build palettes of custom user-interface objects and drag and drop them or connect them, just like Apple's own built-in objects (like NSTextFields, NSTables, NSPanels, etc).

So, if I want to change anything on Luca's interface while developing on Lion, I sinply can't because I can't open any of my nib files due to their dependencies on my custom interface objects (in Luca's case, these objects dealt with the formatting of currencies whenever Luca presents money values in its windows). In Apple's scheme of things, I had to find a machine that is still running Snow Leopard, and run the previous Xcode 3.2.x on it and then open Luca and cut off all dependencies on the custom nib objects.

Theoretically, I could open Xcode 3.2 on Lion but, in reality, installing Xcode 3.2 on top of Xcode 4.1 would mangle Xcode 4.1. I took a long time to settle in on my Lion development - I didn't want to start all over again. So, fortunately, my wife's MacBook Pro was still running Snow Leopard. So I re-installed Xcode 3.2.6 on it. And killed off the dependencies and rewrote some code to replace the lost functionality.

This is really like taking a few steps back. But developing for Apple is often like that. You have to run just to keep in place.

But then, I've always been a Mac fan. I can buy in to what Apple would like to do to keep their platform alive going forward. But I wonder. Would we think back, years later, to Mac OS X Snow Leopard as being the zenith of Mac development, where we can make the Mac do almost absolutely whatever we can dream of, before it descended into this walled (curated?) garden that is the iOS?

MailServe and DNS Enabler for Lion Updates

With MailServe for Lion 5.0.1, I updated Dovecot to its current latest version, 2.0.13. I also fixed two problems with Dovecot. Dovecot can now allow SSL ports (995 and 993) to be opened without requiring the corresponding default ports (110 and 143) to be opened, too (so that you can force a user to connect to Dovecot only via SSL).

Also, this update restores Dovecot's ability to support custom port numbers for POP and IMAP. Please remember to save your config, do a de-install from MailServe's Help menu, re-open the saved config, and then restart all the services, in order to get these latest Dovecot binaries.

It's currently very hard to work with Dovecot 2.0.x because there is just no documentation to be found. I found the workarounds to these problems (plus that about how to use an Intermediate CA SSL cert) by Googling for the answer and not via the Dovecot Wiki. But I don't want to knock Dovecot, which is really a very good system.

For DNS Enabler for Lion 5.0.1, I fixed a problem whereby the dyndnsupdate.key used to support dynamic updates cannot be found on a Lion system with a case-sensitive file system.

So, it's really not that easy to build and support these systems because there are so many variations to test and consider.

MySQL & PostgreSQL Database Installers for Lion

I've uploaded new Lion versions of our database installers - for MySQL and PostgreSQL. They come with their own Startup Preference Panes that you can install into System Preferences. I've tested them. They work great.

The MySQL version this installs is 5.1.57. We'll want to build a version for MySQL 5.5.x and get everything we do aligned with that.

The PostgreSQL version is the current latest, 9.0.4. There's a PostgreSQL bundled within Lion but I haven't figured how to turn that on. But there's an advantage to using our own because I think we can keep up with the changes at Postgres faster than if we depend on Apple and our installer upgrades our database data automatically with each update. And, finally, we can get rid of our own PostgreSQL setup cleanly when we don't need them.