Python file handling

A lot of Python tinkering today, mainly in terms of file naming, management and extracting the contents. First off, how to create sensible file names: Date, Time, Details, Extension. So the code for this looks as follows:

from time import localtime
datetime_stamp = '%4d-%02d-%02dT%02d-%02d-%02d' % localtime()[:6]
title = "TestLogFile"
ext = "log"
print "Unique filename: %s-%s.%s" % (datetime_stamp, title, ext)

Which gives the following type of output:

Unique filename: 2011-10-31T10-09-33-TestLogFile.log

Tab delineated data can be read using the csv module as shown:

# for tab delineated ('\t')
import csv
data = []
for row in csv.reader(open(filepath), delimeter='\t'):
    data.append(row)

header = data[0]
values = array(data[1:])

To get your data into a csv file then the easiest thing to do is to use some form of the following code:

# create csv
import csv
L = [["Time", "Value", "Notes"], [0, 20, "Start point"],\
[1, 0.1, "Mid point"], [2]]
f = open("C:\\Users\\al\\Desktop\\test.csv", "wb")
csv.writer(f).writerows(L)
f.close()

Often, dealing with binary data can reduce the filesize and increase the speed of access:

# binary files
from array import *
f = open("C:\\Users\\al\\Desktop\\b.u32", "wb")
b = array('d', range(10))
print b
b.tofile(f)
f.close()

c = array("d")
f = open("C:\\Users\\al\\Desktop\\b.u32", "rb")
c.fromfile(f,7)
f.close()
print c

and then if you want to find out more about all the data that you have on a disk, you can use the following:

# walk a directory and store in an array
import os
def get_all_files(mypath):
    allfiles = []
    for root, dirs, files in os.walk(mypath):
        for file in files:
            pathname = os.path.join(root, file)
            filesize = os.path.getsize(pathname)
            allfiles.append([file, pathname, filesize])
    return allfiles
allfiles = get_all_files("C:\\Users\\al\\Desktop")
print a

This will give you an output of [file, path, size].

More ways to write data are shown here:

grocery = "Milk 2\nEggs 12"
f = open("C:\\Users\\al\\Desktop\\tobuy.txt", "wt")
f.write(grocery)
f.writelines(grocery)
f.close()

and to read data are here:

f = open("C:\\Users\\al\\Desktop\\tobuy.txt")
text = f.read()
print text
f.close()
print open("C:\\Users\\al\\Desktop\\tobuy.txt").read()
print open("C:\\Users\\al\\Desktop\\tobuy.txt").readlines()

for i, line in enumerate(open("C:\\Users\\al\\Desktop\\tobuy.txt")):
    print "%d: %s" % (i, line.rstrip())

And finally, to search for a substring in a large data file and print out the line number and line:

# Search in a file
def searchfile(filename, substr):
    """Searches for a substring"""

    for i, line in enumerate(open(filename, "rt")):
        if line.find(substr) != -1:
            print "%5d: %s" % (i, line.rstrip())

searchfile("C:\\Users\\al\\Desktop\\tobuy.txt", "Eggs")
Advertisements
Tagged ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: