All software has bugs. I rarely feel scared because of a bug – even famous ones that are sometimes in the news. People should manage software wisely, and be able to workaround bugs they hit.
Instead, I’m more concerned about how vendors treat their bugs. In particular, I care about transparency, and how projects process their bugs.
Here’s what I mean.
Transparency:
- Ability to search for new, active, and closed bugs.
- Public tests to check yourself if bugs are fixed, and if regression bugs made their way back into the code.
- Public version maintenance policy – if I use an old version, I should know if bugfixes are still backported.
- Optional: See developers comments, see their activity on a specific bug.
- Optional: See which version should contain a fix (if planned) and when that version will be released (if planned).
I want to highlight the word Optional. Points 1, 2 and 3, in my opinion, should be in place for any project that is not a personal hobby. If they miss (quite often, I have to say…) the project can still be useful and cool, but it is not being managed properly from QA point of view.
Bug processing:
I am not able to write a detailed list here. I think that this part depends on project characteristics: what the program does, if it is led by a company or not, how active the community is in development, and so on. Still, common sense should apply. Users are affected by bugs, therefore bugs should be fixed when it is reasonable to do so.
When it comes to open source RDBMSs…
Yes, I’m only talking about relational databases. And I’m only consider the most widely known ones. With this, I don’t want to disrespect any project. There can be great reasons to use NoSQL or less known SQL databases, but NoSQL is a huge set of different products, and I only want to write about what I know (or “know”).
MySQL
MySQL has a public bug tracker, which is good. But they also have a private one, and this has some bad consequences:
- If you read about a bug that is still active, you cannot be sure that information is up to date.
- Some bugs are private, for security reasons. Those reasons are often quite obscure, if not always. Note that MySQL typically runs on open source layers whose security bugs are public.
- For the same reasons, not all tests are public. This reduces community QA capabilities.
But you can read more detailed information about this topic in Blog of (former?) MySQL Entomologist.
Another flaw is that not all MySQL components have the same level of quality. If you make heavy use of certain features (XA transactions, stored procedures…) you can reasonably expect to hit bugs. This is partly understandable, because Oracle needs to improve the features that are most used and useful. What is much less understandable is how an area of the product that was disappointing and buggy many years ago is still disappointing and buggy.
About the bug tracker… I’d like to highlight something that is rarely mentioned: its search capabilities are powerful. Maybe not so user friendly – you need to know how to use the tool. But once you know, you will be able to find everything you need.
Percona Server
Percona Server is a fork of MySQL which tries to keep the differences as small as possible. Thus, generally MySQL bugs are also Percona Server bugs, and all the features they implement could cause some specific bugs.
If they fix MySQL bugs, you will find this in their documentation.
Specific bugs can be found in Percona Server JIRA.
You can usually find internal details and tests. You know who works on a bug and when. Generally you don’t know in advance which version will contain a fix.
I think they do very well, and no, the fact I worked for Percona does not affect my opinion. Keep in mind, however, that Percona is a customer-centric company. This means that generally they fix bugs that affect their customers. And, obviously, they merge bug fixes that happen in mainstream MySQL code.
MariaDB
They also have a JIRA to report and search bugs. In their JIRA you can also find tests and several technical information written by the developers. It happened to me more than once to read more information about a bug and find interesting details about InnoDB internals.
In MariaDB JIRA you can also find some information that you can’t find in Percona JIRA. You can see the active sprint’s activity, so you know which bugs they are working on and what their status is (todo, in progress, in review, done). And you can see the next releases, including the next planned release dates (keep in mind that they are not commitments). This is a very high level of transparency and potentially a valuable source of information, for example if you are planning upgrades or if you have pressing problems with a specific bug. And of course it’s still interesting if you are just curious.
I also want to highlight that they discuss in the same way contributions and new features. Also, developers often interact with the community using their mailing lists, which makes the whole project much more transparent.
PostgreSQL
PostgreSQL does not have a bug tracker. Bugs can be reported to the dedicated mailing list or via a web form which sends a message to the mailing list. The archives are the only available tool to see information about Postgres bugs. In other words, there is no way to see the list of open bugs. This is sad.
However, Percona recently started to offer support for PostgreSQL. Their Careers page indicated that they were looking for someone able to work with Postgres code, some time ago (yes, I keep an eye on some companies career pages, because they are a source of information about their plans). Therefore my hope is that they intend to do a great QA work with Postgres like they always did with MySQL, and that they will push to have a publicly available professional bug tracker.
CockroachDB
CockroachDB uses GitHub issues to track bugs. The tool is quite poor compared to JIRA or bugs.mysql.com, but it has the most important features that you can reasonably expect.
Usually there are not many technical details about bugs. And you don’t know what is happening around a bug, unless there is a new comment.
Why do I always criticise?
Actually I don’t. But I’m aware that, if a person expresses 3 positive thoughs and 1 criticism, in many cases only the latter will be remembered. This is natural.
Anyway, my spirit is always: please do better. I use my time to criticise what I like, not what I dislike.
My criticism could also be wrong, in which case there could at least be a good discussion in the comments.
Federico
FALSE IS NOT NULL 