February 14, 2010

Gerald release 0.3.6

Filed under: database, python — Andy Todd @ 2:06 pm

I have just released version 0.3.6 of Gerald. Gerald is a general purpose database schema toolkit written in Python.

This release was at the request of the sqlpython project and contains only one change. A new convenience method connect has been added to the Schema class. This enables a schema to be initiated and then later have a database connection associated with it. Because this changes the public API of gerald I’ve released this under a new version number.

Development, bug and issue tracking and the project wiki are available on the project Trac site. Source code and distribution files are available at the sourceforge page.

The next release will be 0.4. Exactly what will make up that release is still evolving, although it is likely to feature SQL Server support as I have just started a new job and all of the systems there use it. To see what else is in the release and to track progress take a look at the version 0.4 roadmap.

January 21, 2010

Gerald release 0.3.5

Filed under: General, database, python — Andy Todd @ 9:40 am

This last weekend I released version 0.3.5 of gerald.

The major component of this release was to add a ‘User’ class to the oracle_schema module. This is similar to the ‘Schema’ class but whilst that shows all of the objects a database user owns the ‘User’ class contains details of all of the objects they can access, including those owned by other database users. This was requested by the sqlpython project to enable them to use gerald for database introspection.

The only other change was to ensure that the NotImplementedError exception is raised in all of the super type methods that are just stubs. This is mainly in the Schema.py module and thus meant that I had to add a set of tests for this module.

Development, bug and issue tracking and the project wiki are available on the project Trac site. Source code and distribution files are available at the sourceforge page.

The next release will be 0.4. Exactly what will make up that release is still evolving. To see what is in the release and to track progress take a look at the version 0.4 roadmap.

November 25, 2009

Gerald release 0.3.1

Filed under: database, python — Andy Todd @ 3:25 pm

Everyone, say hello to version 0.3.1 of gerald. This is a minor update that fixed some issues introduced in release 0.3 In summary these are:

  • Ticket 17 – Views have been converted to dictionaries from tuples
  • Ticket 18 – Reading an Oracle sequence updates it’s current value
  • Ticket 19 – Postgres primary keys were not represented properly when read from the database

Development, bug and issue tracking and the project wiki are available on the project Trac site. Source code and distribution files are available at the sourceforge page.

The next release will be 0.3.5 and will introduce the concept of a ‘User’. This is similar to a ‘Schema’ but will reference all of the objects a database user can see even if they don’t own them. You can track progress for the release using the version 0.3.5 roadmap.

May 4, 2009

Gerald release 0.2.6

Filed under: database, python — Andy Todd @ 12:38 pm

I have packaged and released version 0.2.6 of gerald. This is an update that doesn’t change any functionality but has involved a major re-factoring of the module unit tests. Rather than wait I have released the package in it’s current form to give a base for some major changes coming in release 0.3

Gerald is being considered for use in the SQLPython toolkit. For them to get full value I will need to make a number of changes to the code. Luckily for all of us these were on the development roadmap anyway, so I’ve just moved them up the priority list.

As other people have shown an interest in the tool I have also invested in a little infrastructure and have fired up Trac for the sourceforge project. Gerald now has a wiki and, more importantly, a ticketing system for bug reports and feature requests. I’ve put the changes coming for release 0.3 in a number of tickets and you can track progress for the release using the version 0.3 roadmap.

April 19, 2009

Gerald release 0.2.5

Filed under: database, python — Andy Todd @ 3:06 pm

I have packaged and released version 0.2.5 of gerald. This is another minor release with a couple of bugs fixes and one important administration update. You can find all of the details in the CHANGELOG.txt file that comes with the source distribution (or view it here) but in a nutshell the changes from the last release are;

  • Fixed a bug in some connections to PostgreSQL
  • Removed references to database links without passwords in Oracle
  • Converted the package to use setuptools and from this release onward it is installable using easy_install

The last change was at the request of the SQLPython project, if you haven’t taken a look at that wonderful utility yet, please do. To support them it is likely that the next release of Gerald will support SQLite.

Please download, install and enjoy. Bug reports. tumultuous praise or cash bribes to the usual address please.

January 16, 2009

Web 2.0 – better late than never

Filed under: database — Andy Todd @ 8:53 am

Just to prove that I am doing things even when I’m not posting here I am leaving my mark on the internet at these other places as well;

  • My tumblog – is that even what they are called?
  • Twitter – of course
  • Plurk – although these messages may be suspiciously similar to my tweets
  • Facebook – just to keep up with the juvenile party photos of people I went to university with
  • Flickr – although most of the photos are restricted, if you want to see them drop me an email.

And you never know, I may actually post something technical here in the near future.

January 3, 2009

Gerald release 0.2.4

Filed under: database, oracle, python — Andy Todd @ 3:17 pm

I have packaged and released version 0.2.4 of gerald. This is a minor release with a couple of little features and some documentation and admin updates. You can find all of the details in the CHANGELOG.txt file that comes with the source distribution (or view it here) but in a nutshell the changes from the last release are;

  • Added support for comments on columns in Oracle
  • Added support for comments on tables in Oracle
  • Changed the stylesheet used for documentation
  • Added more information to the README file and converted it to reStructured Text format

Please download, install and enjoy. Bug reports or tumultuous praise to the usual address please.

December 9, 2008

Oracle Parameter Files

Filed under: database, oracle — Andy Todd @ 3:07 pm

This isn’t going to be news to any Oracle DBAs reading this. So if you are one, can I suggest you move along quickly to the next item in your feed reader.

Anyone left is hopefully curious like I was to figure out the difference between ‘traditional’ oracle initialisation files (init.ora) and the new fangled server parameter files (sp files). Well wonder no further fair reader for I can explain all.

Prior to Oracle 9.2 (a long time ago in Oracle database years) it was only possible to permanently effect the configuration parameters of your database by changing the init.ora file and restarting your database instance. You could make changes whilst the database was running by using the ‘ALTER SYSTEM SET’ command but these would be lost the next time your database was restarted because the startup process would use the values in your init.ora file. This meant that any instance tuning would require some informed ‘twiddling’ by the DBA to settle on a suitable value for an instance parameter and that would then be followed by a restart, causing anyone using the database to be mightily inconvenienced.

With the introduction of server parameter files in 9.2 and above changes made interactively (using ‘ALTER SYSTEM SET’) are automatically persisted. This is because whenever a change is made this way it is automagically written to your spfile.

This article has a much more comprehensive description of the differences between the two different types of files. One of the useful things it mentions is the resolution order the database server uses to select an initialisation file on startup. This is;

  • spfile<SID>.ora
  • spfile.ora
  • init<SID>.ora

Tick another thing off the list of ’stuff that has been bothering me and I need to figure out’.

August 4, 2008

Gerald release 0.2.3

Filed under: database, oracle, python — Andy Todd @ 7:08 pm

I have packaged and released version 0.2.3 of gerald. This is mainly a tidy up and bug fix release, with numerous additional unit tests. You can find all of the details in the CHANGELOG.txt file that comes with the source distribution (or view it here) but in a nutshell the changes from the last release are;

  • Numerous renames to more fully comply with PEP-8
  • Added numerous tests, full details in the CHANGELOG.txt file
  • Added an ‘enabled’ flag to table constraints (except in the MySQL module, because that database doesn’t support them)
  • Added support for column defaults in Oracle
  • Added a ‘table_name’ attribute to the schema.Trigger class
  • Removed main and usage functions from each module that shouldn’t be imported

Please download, install and enjoy. Bug reports or tumultuous praise to the usual address please.

May 9, 2008

Trouble Getting a Date

Filed under: database, python — Andy Todd @ 8:04 am

I’m having trouble with dates. This can be summed up in a couple of high level issues;

1. Date support in relational databases is insane, or at the best inconsistent.

As far as I can tell the ANSI SQL-92 standard defines date, time, interval and timestamp data types. Which doesn’t help when SQL Server only implements something called ‘datetime’ – at least I think so, have you tried accessing any sort of manual for a Microsoft product online? Blimey, I thought billg had embraced this web thing years ago. Oracle has the ‘date’ data type (which is actually a time stamp) and MySQL, well they’ve gone and outdone everyone by implementing DATETIME, DATE, TIMESTAMP, TIME, and YEAR.

2. The Python DB-API does not cope with date data type ambiguity well.

When it comes to the date question the Python DB-API states (and I quote) ” … may use mx.DateTime”, which if you ask me isn’t much of a standard. This needs to change so that all DB-API modules return consistent datetime objects, not such a big issue as datetime has been part of the standard library since, what, Python 2.3?

Sadly even if we fix this it won’t work with Sqlite as it doesn’t consistently support data typing. In my experiments regardless of what sort of date you insert into the database you get a unicode string back. Don’t believe me? Try this in Python 2.5;

>>> from sqlite3 import dbapi2
>>> db = dbapi2.connect('test_db')
>>> cursor = db.cursor()
>>> cursor.execute('create table date_test (id integer not null primary key autoincrement, sample_date DATE NOT NULL)'
>>> stmt = "INSERT INTO date_test (sample_date) VALUES (?)"
>>> cursor.execute(stmt, (1234, ))
>>> import datetime
>>> cursor.execute(stmt, (datetime.date(2008, 3, 10), ))
>>> cursor.execute(stmt, ('My name is Earl', ))
>>> db.commit()
>>> cursor.execute("SELECT * FROM date_test")
>>> results = cursor.fetchall()
>>> for item in results:
...     print item[1], type(item[1])
1234 
2008-03-10 
My name is Earl 
>>>

But note that it is fine for integers.

3. The people writing the Python standard library modules are on crack.

Outside of the database world and within the batteries included Python standard library some modules use datetime, others time and there are even uses of calendar.

O.K. I’ll accept that maybe the module authors aren’t on full strength crack, because the time module just exposes underlying posix functions. But the people who wrote those were on something strong and hallucinogenic. I table the following function signatures from section 14.2 of the Python Library Reference 2.5 as an example;

strftime(format[, t ])
strptime(string[, format ])

This has bitten me twice in the last twenty four hours and frankly I’m not happy.

I appreciate that there are historical reasons for having inconsistent function signatures but can someone please fix this in Python 3.0. All we need is a single module that can access the underlying system clock and then convert between a number of different representations of that and other epoch driven dates. How hard can it be? As far as I can tell this is not part of the proposed standard library re-organisation. I think it should be.

Next Page »

Powered by WordPress