Tag Archives: Python

Rasterio conflict in Anaconda

Based on a comment from ocefpaf at https://github.com/conda-forge/gdal-feedstock/issues/69

I recently had an issue where my Anaconda environment was having issues when I wanted to import gdal or rasterio. I was getting the following error when I tried to import rasterio:

ImportError: libmfhdf.so.0: cannot open shared object file: 
No such file or directory

It seems to be a reasonably common issue based on online searches. The following fix worked for me and I was able to install other packages that I needed into the new env and it (so far) seems to be working OK.

conda create -n rasterio_test_env python=3.5 rasterio --yes -c conda-forge 
source activate rasterio_test_env
python -c "import rasterio; print(rasterio.__version__)"

 

Advertisements
Tagged

Processing set up

This post is here as a reminder of what I did to setup my coding environment. If it is of any use or interest to anyone else then that’s great 🙂

First up I set up a VirtualBox VM on my desktop (from Entroware!). It is running an Ubuntu Mate 16.04 guest, but I guess that is inconsequential, as it could be running any version. It has the VirtualBox Additions installed, NAT and Bridge Adaptor networking are enabled, and a shared folder on the host has been setup.

On the guest, the shared folder is mounted using the command:

sudo mount -t vboxsf -o uid=1000 sharedVMfolder path/to/mountpoint

Anaconda Python has been installed on the guest and a new environment called spatialP3 has been created using:

conda create --name spatialP3

This is activated using:

source activate spatialP3

In that environment, a number of spatial libraries have been installed using:

conda install --name spatialP3 packagename

There were a number of version conflicts with some of the default packages, so I searched the Anaconda Cloud to find the most recent packages and installed them from the Anaconda Cloud repository using:

conda install -c conda-forge shapely=1.5.16

and similar.

To launch a Jupyter Notebook server with no local display (i.e. on the guest) use:

jupyter notebook --ip=0.0.0.0 --no-browser

Port forwarding has been set up in Virtualbox using the TCP protocol and with the host port being 8899 and the guest port being 8888 (the default for Jupyter Notebooks).

This means that you can run the notebooks from the host (as long as the VM and Jupyter server are both running) by typing the following into your browser:

localhost:8899

or from any machine on the network by typing in the local IP address of the host machine and the port 8899. If you start them in the correct location i.e. the shared folder that was mounted in the VM, then the notebooks will be accessible irrespective of whether the VM is running, and can be part of a backup strategy for the host machine.

Tagged , ,

‘Fixing’ GRASS

I use Ubuntu as my operating system and have GRASS GIS installed through the ubuntugis-unstable repository. Everything has been working great. Then I installed the landsat-util tool and all sorts of ‘stuff’ started getting installed and updated. I know I should have kept an eye on what it was going to do but I was in a rush and just said ‘Y’ whenever prompted. Bad practice. Anyway, the upshot was that landsat-util installed and worked perfectly, but GRASS GIS fell over. It would not load the GUI. Damn!

The message I was getting was something along the lines of

This module requires the Numeric/numarray or NumPy module, which could not be imported. It probably is not installed
(it's not part of the standard Python distribution). See the Numeric Python site (http://numpy.scipy.org) for information on
downloading source or binaries.
ImportError: Numeric,numarray or NumPy not found.

When I checked in Python, I had numpy installed, so that wasn’t the issue.

I couldn’t find anything useful online to help me fix it but then I thought, it’s Python and it’s open source. Let’s have a hack!

I opened the following file (if I was more leet I would have used vim or nano or something):

sudo geany /usr/lib/python2.7/dist-packages/wx-2.8-gtk2-unicode/wx/lib/plot.py


I then went to line 117 and added the first three lines shown in the code block below, remembering to alter all the indents etc:

try:
    import numpy as _Numeric
except:
    try:
    import numpy.oldnumeric as _Numeric
    except:
        try:
            import numarray as _Numeric #if numarray is used it is renamed Numeric
        except:
            try:
                import Numeric as _Numeric
            except:
                msg= """
This module requires the Numeric/numarray or NumPy module,
which could not be imported. It probably is not installed
(it's not part of the standard Python distribution). See the
Numeric Python site (http://numpy.scipy.org) for information on
downloading source or binaries."""
                raise ImportError, "Numeric,numarray or NumPy not found. \n" + msg

 

I then saved the file and restarted GRASS and it all (seems) to be running smoothly and as it should do.

Disclaimer: Now I’m no programmer, just a hacker of code and general tinkerer. Your mileage may vary, but if this is of use to anyone then that’s great. It’s of use to me and that’ll do me fine.

Tagged , ,

Python file handling

A lot of Python tinkering today, mainly in terms of file naming, management and extracting the contents. First off, how to create sensible file names: Date, Time, Details, Extension. So the code for this looks as follows:

from time import localtime
datetime_stamp = '%4d-%02d-%02dT%02d-%02d-%02d' % localtime()[:6]
title = "TestLogFile"
ext = "log"
print "Unique filename: %s-%s.%s" % (datetime_stamp, title, ext)

Which gives the following type of output:

Unique filename: 2011-10-31T10-09-33-TestLogFile.log

Continue reading

Tagged ,

Numpy

I have spent the last day or so looking at the interaction between PIL and numpy. PIL is useful for quickly loading and manipulating images (size, subsets etc) but if you really want to get to grips with image processing you need to understand numpy and the array structure. If you have come from Matlab then it can be a bit confusing as the syntax is “similar but different”. Anyway, here are some basic snippets of code:

import Image as I # Import PIL
import numpy as N # Import Numpy
import pylab as P # Import Matplotlib functions
a = I.open("/home/al/Documents/temp_image_proc/bird/image4.jpg", "r") # Open the image
b = N.asarray(a) # Convert to Numpy array
red = b[:,:,0] # Extract red band
P.imshow(red) # Display the image
P.show()

So we import the Image functions from PIL, numpy and the matplotlib plotting functions in pylab. Then we open up an image (.jpg format) convert it to an array, subset out the red band and display it. The image is shown here:

Continue reading

Tagged , , , , ,