First let me apologize for dropping off the face of the earth for the last 3-4 weeks. Paying work got a little nuts and then we ran into our annual user group meetings for those same customers. Between those, that pretty much killed a month.
That said, it really turned out to be a net positive, because it gave me a bit of time away from the current state of affairs to really sort out where we need to go with this. To really gnaw on some of the core issues that surround this project in specific, but also PostgreSQL as a whole. It also let me sort out where I think this project needs to go with regards to those issues.
Earlier this year, I created a PostgreSQL steering committee. Ultimately, they will help make some of these decisions, but I am going to lay out my thoughts here and now, and we can all take it from there. First we need to tackle the issues (as I see them).
The Issues
Lack of external tracking of issues
The strength of PostgreSQL may well be one of it's greatest weaknesses. The distributed 'work on what you want' nature of Open Source development means that sometimes issues can sit for a long time before they get addressed. To make matters worse, PostgreSQL has elected not to use an issue tracking system like Bugzilla, but has instead chosen to use a mailing list for tracking bugs. This can make it tricky to filter out what is or is not being worked on. I know of a couple of issues that I have elected not to work on because I believed them to be in process based upon -bugs and -hackers mail traffic. One has existed for a while and has not been addressed. I know that those effected are looking at alternative platforms to PostgreSQL because they have no way of knowing if an issue is being worked on.
This means that people that are not active on both -bugs and -hackers have no way to 'track' or 'watch' issues once they have been reported. It also means that some of us that are not full-time on the PostgreSQL mailing lists sometimes have difficulty monitoring the status of a particular issue. I do not see this changing within the PostgreSQL.org environment.
Granularity of Focus
This is not a PostgreSQL issue so much as it is an issue with this project. The initial goal and mindset of this project was very narrow. Build PostgreSQL with the most common options and configure it so that it fit naturally into the Mac OS environment and ecosystem. I think we accomplished that, and then the scope grew into the GUI tools, and then the developers frameworks. Somewhere along the we, that narrow focus either got lost or misplaced.
Apple's Installer is not well suited to the PostgreSQL for Mac Project
Do not misunderstand, the Installer.app is a nice tool, but for a project that has so many updates, and such a massive base of potential add on modules, it is not well suited to the need. What is needed is something that has more in common with the package managers or Sparkle than Installer.app.
No central repository for the modules and addons for PostgreSQL
This is an old problem. There is a repository in pgFoundry, if you build from source. You can also use DarwinPorts or Fink to install everything, but there are issues with all of the above. Not the least is that most servers do not have the Developer Tools installed on them. It also means an additional learning curve for many people that are not prepared to, or are outright afraid to tackle.
Dealing with the Issues
There is no silver bullet here, and honestly we are a very small fish in the pond, even in the smaller Mac Using PostgreSQL pond. So the scope of what we *can* tackle is relatively limited. Acknowledging that, it occurs to us that what we can do is tackle each of these topics in our little corner of the pond.
Lack of external tracking of issues
Unfortunately, there are limits to what issues we can tackle in the trunk code base. What we can tackle though, we can track. Though I am not the worlds biggest fan of SourceForge, that is where we set up shop for PostgreSQLforMac 6 years ago. It makes no sense to go elsewhere at this point, despite my own preference for a JIRa/Confluence/Bamboo configuration. We can start tracking issues that we are either 'tracking' on -bugs and -hackers or are working internally. This gives our customers a place to go o track issues that have been reported directly to us.
(Francois, if you would resubmit your history on your Collation bug, we think we would like to start with that one)
We know that this will never percolate to the larger PostgreSQL community, the discussions there regarding bug tracking systems are just not worth rehashing. Suffice it to say, they will choose to do what they choose to do.
Granularity of Focus
This is really a tough one. The truth is, we would really like to be a one stop shop for all things PostgreSQL on the Mac. This would require a time commitment that our PostgreSQL derived income simply cannot support. Understanding this, we think that solving the next issue will actually help with this while making the real desire a little bit more attainable. In the short term though, the focus almost has to go back to the original goal, along with a smaller subset of the tools to make it all work. The other fluff pretty much has to go back being paid for work until the total revenue is sufficient to support it. Sadly, it comes back tot he 'making money on Open Source is tough' comments that you are seeing in mainstream tech media right now.
Apple's Installer is not well suited to the PostgreSQL for Mac Project & No central repository for the modules and addons for PostgreSQL
This one, and the next one are tightly wound together, so we are 'bundling' them together. The solution to one, quite naturally links to the other.
The more we fight with Installer.app and how it overwrites directories, the more evident it becomes that Installer.app just doesn't work for what we want to do. Before we get into the details, we need to give some background.
If you have looked inside the /PostgreSQL8/ folder lately, you will have noticed that it's internal structure is built to support multiple versions in the same folder that can be 'switched' at the users discretion. This idea is to allow developers and deployers both to easily switch between minor versions without a major hassle of backup/uninstall/reinstall/restore. We would really like to also make it trivial to do the same with the various 'addon' modules that PostgreSQL supports (think PostGIS as a 'for instance').
We have spent months trying to work within the limitations of Installer.app to accomplish this, and yet the harder we push it, the worse it behaves.
For that last couple of months, we have been tossing around the idea of doing something else for the future installs. The 8.4.3 installer failures have only pushed this idea harder. After a few weeks away, I am sold.
What we want to do is add a PostgreSQL Installation Manager to the distribution. Using it, you can install a PostgreSQL version from a locally stored package, or from a Server based package that we provide. The process works like this:
A. User downloads the 'unified installer'. It installs the installation manager and a small collection of local packages (current version, gui tools, etc). Once complete, it allows the user to select add ons that may be wanted, directly from the PostgreSQL for Mac Repository. When a new version is released, it works like Sparkle, enabling a self update, while retaining your current version. You can then choose to 'test' a new version and roll back quickly if it doesn't work out. When you are done with a version, you can easily remove it.
Yes, this is a ton of work. In the long run though, it provides a blueprint for something that can be the centerpiece of the best Mac Database solution on the market, and that is what we want.
But what does that mean in the short term? well, it means that a fixed 8.4.3 has to be dealt with. Probably this weekend. It also mean that 9.0 is the earliest we would see this. There are a couple of other changes that have to be dealt with for 9.0 anyways, but that is a topic for another book :).
Regardless, we are not going to tackle this until we get some thoughts from you the users, so please post up, or email me. We need to work on some design stuff, at this point everything is pie in the sky concept stuff.