15th April, 2014

Generating Reasonable Passwords with Python

Filed under: python — andy47 @ 4:33 pm

Thanks to a certain recent Open SSL bug there’s been a lot of attention paid to passwords in the media. I’ve been using KeePassX to manage my passwords for the last few years so it’s easy for me to find accounts that I should update. It’s also a good opportunity to use stronger passwords than ‘banana’.

My problem is that I have always resisted the generation function in KeePassX because the resulting strings are very hard to remember and transcribe. This isn’t an issue if you always use one machine but I tend to chop and change and don’t always have my password database on the machine I’m using. I usually have a copy on my phone but successfully typing ‘Gh46^f27EEGR1p{‘ is a hit and miss affair for me. So I prefer passwords that are long but easy to remember, not unlike the advice from XKCD.

Which leaves a problem. Given that I now have to change quite a lot of passwords how can I create suitably random passwords that aren’t too difficult to remember or transcribe? Quite coincidentally I read an article titled “Using Vim as a password manager”. The advice within it is quite sound and at the bottom there is a Python function to generate a password from word lists (in this case the system dictionary). This does a nice job with the caveat that it I understand from a cryptographic standpoint the passwords it creates are not that strong. But useful enough for sites which aren’t my bank or primary email. For those I’m using stupidly long values generated from KeePassX. When I tried the Python function on my machine there was one drawback, it doesn’t work in Python 3. This is because the use of ‘map’ is discouraged in Python 3. But that’s alright because I can replace it with one of my favourite Python constructs – the list comprehension. Here is an updated version of invert’s function that works in Python 3. Use at your own risk.


def get_password():
    import random
    # Make a list of all of the words in our system dictionary
    f = open('/usr/share/dict/words')
    words = [x.strip() for x in f.readlines()]
    # Pick 2 random words from the list
    password = '-'.join(random.choice(words) for i in range(2)).capitalize()
    # Remove any apostrophes
    password = password.replace("'", "")
    # Add a random number to the end of our password
    password += str(random.randint(1, 9999))
    return password

10th January, 2014

It’ll Come Out in the Wash

Filed under: python — andy47 @ 3:36 pm

After reading this fine summary of the history of Python 3 by Nick Coghlan I was inspired to update as many of my half finished projects and miscellaneous scripts as possible. Then I looked up and I had lost several hours of my life. To save random internet strangers from the same pain as I experienced here is a catalogue of problems and how I solved them.

tl;dr – Python 3.3 on Ubuntu 12.04 LTS is possible but quite tricky. Unless you like system administration you are probably better off waiting until 14.04 LTS comes out in a few months.

I’ve settled on a default ‘stack’ of Python and related technologies that I use for most of the code that I write these days;

The good news is that all of the Python related components of these technologies have been ported to Python 3. The bad news is that Flask only runs on Python 3.3. Elementary OS is based on Ubuntu 12.04 LTS and that *only* has Python 3.2 in it’s repositories.

So, first things first, how do we get Python 3.3 on Elementary OS? Or Ubuntu 12.04 which are interchangeable for the purposes of the rest of this post. Quite easily thanks to the ‘deadsnakes PPA’ as explained in this askubuntu post.

I also don’t like to pollute my system Python or operating system packages with project specific Python modules so I always use virtualenv. Unfortunately this doesn’t work with the Python 3.3 acquired from the deadsnakes PPA as *that* doesn’t ship with easy_install.

So by this stage I have Python 3.3 running but no way to create or manage a virtual environment. I then remembered Nick’s talk at PyCon AU where he mentioned that a new lightweight virtual environment would be included in the standard library. Sure enough the venv module (and pyvenv script) were present on my system so I decided it was a good idea to use them.

The bad part for me is that (and it says this in the documentation) venv does not include any means of installing other Python packages. How then do you get a package installer/manager when you can’t install packages? The solution was in a c.l.p. post.

That link describes a process of creating a new virtual environment and then bootstrapping distribute. In case my link-fu fails me the simple shell based approach is;

$ pyvenv py3k ~/envs/py3k
$ cd py3k
$ source bin/activate
$ wget http://python-distribute.org/distribute_setup.py
$ python distribute_setup.py
$ ./local/bin/easy_install pip

One final little wrinkle though. The c.l.p. post and other references I found indicate that your package scripts (such as ‘pip’ and ‘python’) will be installed in $ENV_HOME/bin. In my environment they are installed in $ENV_HOME/local/bin. It’s probably something I have inadvertently done. But if you follow the instructions and can’t find something in bin have a look in local/bin and you should find it.

22nd November, 2012

Living in the Future

Filed under: python,ubuntu — andy47 @ 11:24 am

On my morning commute today I realised that I am actually living in the future. I remember when I got involved in the PythonCard project 10 years ago one of the major questions on the mailing list was why we were building a GUI toolkit when the future was the web. It wasn’t true then but I think that it is now.

Why do I think we have moved now? It is in large part thanks to a book I have started reading called Python for Data Analysis. I have a copy of the book in ePub format and wanted to read it on my laptop. After some research instead of an e-reader I actually installed a web browser plugin called Readium to view the book.

I then wanted to set up an environment for working through the examples from the book. I created a virtualenv on my Ubuntu server based VM and installed the required modules. After a couple of pages I realised that I needed some sort of graphical environment for rendering graphs. Rather than move to a desktop virtual machine I decided to go for another option. I read the documents and fired up an IPython notebook with remote access. The only thing missing from my useful toolset is a VIM instance. I’m sure that can’t be far away.

Or you can try this android smart tv @qway

All of which means that within a single browser (on separate tabs) I am both reading a book and interactively working through the Python code examples from it. I appreciate that there are back end processes involved especially with the IPython notebook. But here in 2012 it is possible to do some amazing things in the browser that I wouldn’t have imagined even a couple of years ago. Did I mention that I really like living in the future?

1st June, 2012

Python path relative to application root

Filed under: python — admin @ 3:53 pm

I’ve recently written some code to wrangle XML files. Part of the code validates a provided file against an XML Schema stored in a file. When I wrote this code I got tangled up in absolute and relative path manipulations trying to load the XML Schema file. Most of the Python file operations work relative to the current working directory and I needed to be able to load my XML Schema from a file relative to the application root directory. Regardless of where the code was executed from the schema file would always be up and across from the directory containing the Python module being executed. A picture will probably help.

Within load_source_file.py I need to load and parse the XML Schema contained in Source_File.xsd. Here’s how I did it. First, we need to work out the root directory of the application relative to load_source_file.py. After a few false starts this tip from StackOverflow was the key – http://stackoverflow.com/a/1271580/2661. The full path to our etl directory is;

root_dir = os.path.abspath(os.path.dirname(__file__))

But we need to go up a directory so we use os.path.split to remove the last component of the path.

root_dir = os.path.split(os.path.abspath(os.path.dirname(__file__)))[0]

The final part is simply joining this with the name of the directory and schema file that we wish to load. Then we have a directory that is the same irrespective of where we run code from. To make reading the code easier I split this across a few lines and ended up with.

>>> import os
>>> from lxml import etree
>>> root_dir = os.path.split(os.path.abspath(os.path.dirname(__file__)))[0]
>>> schema_file = os.path.join(root_dir, ‘schemas’, ‘Source_File.xsd’)
>>> xmlschema.doc = etree.parse(schema_file)
>>> xmlschema = etree.XMLSchema(xmlschema_doc)

10th November, 2011

Extracting a discrete set of values

Filed under: python — admin @ 3:42 pm

Today’s I love Python moment is bought to you by set types.

I have a file, XML naturally, the contains a series of transactions. Each transaction has a reference number, but the reference number may be repeated. I want to pull the distinct set of reference numbers from this file. The way I learnt to build up a discrete set of items (many years ago) was to use a dict and set default.

>>> ref_nos = {}
>>> for record in records:
>>> ref_nos.setdefault(record.key, 1)
>>> ref_nos.keys()

But Python has had a sets module since 2.3 and the set standard data type since 2.6 so my knowledge is woefully out of date. The latest way to get the unique values from a sequence looks something like this;

>>> ref_nos = set([record.key for record in records])

I think I should get bonus points for using a list comprehension as well.

19th August, 2011

Validating an XML File with LXML

Filed under: python — admin @ 8:35 pm

I’ve been playing with XML files recently and have on the odd occasion needed to validate a file against an XML schema. This is surprisingly easy using lxml, the Swiss Army knife of Python XML processing. Allow me to demonstrate.


>>> from lxml import etree
>>> schema = etree.XMLSchema(etree.parse('schema_file_name.xsd'))
>>> xml_file = etree.parse('xml_file_name.xml')
>>> schema.validate(xml_file)
True

Job done. If you are unlucky enough that your file doesn’t validate you can find out by checking the error_log attribute of your XMLSchema object.

 

26th October, 2010

Gerald release 0.4.1

Filed under: database,python — admin @ 11:45 am

Before starting on some of the big changes planned for version 0.5, and thanks to patches and suggestions from various people I’ve addressed a couple of issues with Gerald 0.4. This means that we now have release Gerald 0.4.1

What’s new in this release? Not much, just some bug fixes, documentation changes and (hopefully) an egg that is installable on all platforms. The .egg files available from PyPI (and soon to be available on SourceForge) should install without any errors and if my testing is correct will be usable on multiple platforms including Windows.

Downloads are available at the PyPI page and the SourceForge project page. As always, please send me an email with any problems or suggestions for improvement.

20th August, 2010

Python strftime reference

Filed under: python — admin @ 9:23 am

Take a look at this excellent single page web site – Python strftime reference. It does exactly what it says on the tin. Good work.

27th June, 2010

Gerald release 0.4

Filed under: database,oracle,python — admin @ 3:36 pm

I’ve been revelling in the Python goodness this weekend at PyCon Australia. This has motivated me to fix the last couple of issues and then package and release Gerald 0.4

What’s new in this release? The most important changes are fixes to a number of issues identified by users of SQLPython. Gerald was appearing to take a long time to collect large schemas but was actually failing silently. I added test cases to show the problem and then fixed the code. This shouldn’t happen any more.

I applied a couple of patches supplied by Catherine Devlin to cope with columns without defined lengths and to not get DBA objects in Oracle schemas.

I slipped in some new features as well; I implemented the to_xml and compare methods on the CodeObject class, and Gerald now supports views in MySQL (as long as you are running 5.1 or above).

Finally, I changed the project documentation to use Sphinx.

Downloads are available at the PyPI page and the SourceForge project page.

If you find any problems or want to contribute any code just send me an email.

1st June, 2010

Generating HTML versions of reStructuredText files

Filed under: General,python — admin @ 1:37 pm

I wanted to quickly and easily convert a series of reStructured text documents into HTML equivalents. For reasons too dull to discuss here I couldn’t just use rst2html.py and didn’t want to go to the trouble of remembering enough bash syntax to write a shell script.

So I thought that as long as docutils is written in Python it would only take a moment or two to knock up a script to do what I needed. Well yes, and no. The script itself is fairly simple;

from docutils import core

def convert_files(name_pattern):
    for file_name in glob.glob(name_pattern):
        source = open(file_name, 'r')
        file_dest = file_name[:-4] + '.html'
        destination = open(file_dest, 'w')
        core.publish_file(source=source, destination=destination, writer_name='html')
        source.close()
        destination.close()

The most useful line being the one where I call core.publish_file. But it wasn’t immediately obvious from the docutils documentation what series of incantations would achieve my desired results. Luckily, after some time spent perusing the documents I came across this dissection of rst2html.py. This, in turn, lead me to the description of the Docutils Publisher, which lists the convenience functions available to work with the engine.

The end result isn’t particularly elegant but it does get the job done and I thought I would share it in case anyone else has a similar need in the future.

Next Page »

Powered by WordPress