34. saveload — A new format for saving pyFormex geometry/projects to file.

This module provides some functions to save pyFormex geometry and other objects to a file, and load them back from these files. For this purpose we use a new, experimental file format and the filename extension .PZF (pyFormex zip format).

The new PZF format is experimental, and not complete (yet). Do not use it yet to store data that have to persist over a long period. One day however it could replace the current .PGF (geometry) and/or PYF (project) formats.

The new format is very robust, is easy to implement and extend, provides easy ways of upgrading without losing contents, and guarantees openness to other softwares and portability to other OSes and architectures. It currently stores Formex, Mesh anf TriSurface objects, togerther with the Fields added to the object, and even the drawing attributes.

Basically, PZF files are zip archives written with NumPy’s savez function. Being a zip file, they can be opened with most modern file managers, allowing the user a view on what’s inside. In future we could even provide facilities to extract selected objects from the file and to open an archive directly in pyFormex by just double clicking on the filename. It also provides possibility to compress and even password-protect the PZF contents.

Since most information in pyFormex objects is stored in NumPy arrays, the choice of numpy.savez to write a whole set of arrays to a zip file saved us a lot of own development, and guarantees future support and stability of the format. However, numpy.savez only saves arrays, and we need to store more things for pyFormex objects: the class name, some non-array attributes (usually strings) like ‘eltype’, and the ‘fields’ and ‘attrib’ attibutes. Fields are again mostly an array plus some strings, but ‘attrib’ is a dict that could be quite complex, although it usually isn’t. And the complex things usually can be avoided: instead of setting ‘color=some_large_color_array’ in the attributes, one can add ‘some_large_color_array’ as a Field to the object, with ‘some_field_name’, and then set a drawing attribute ‘color=fld:some_field_name’.

Being a zip archive, the contents of it are individual files. Every file contains exactly one numpy array. We encode some of the string data into the file names: the object name, its class name, the attribute name. For a field we also encode the field name and field type. For example, the contents of the ‘saveload.pzf’ file in the pyformex/data folder is:

F__Formex__coords.npy
F__Formex__prop.npy
M__Mesh__attrib.npy
M__Mesh__coords.npy
M__Mesh__elems.npy
M__Mesh__eltype__quad4.npy
M__Mesh__field__elemc__dist3c.npy
M__Mesh__field__node__dist.npy
M__Mesh__field__node__dist3n.npy
T__TriSurface__attrib.npy
T__TriSurface__coords.npy
T__TriSurface__elems.npy
spiral__PolyLine__attrib.npy
spiral__PolyLine__coords.npy

The .npy extension of the files stresses the fact that each individual file stores a numpy array in .npy format. They could be restored to plain numpy arrays, but the pyFormex reader will take info from the file names and combine them into proper pyFormex objects. The file name is made up of different fields, separated by two underscores ‘__’.

The first two fields are the object’s name and class. Evidently, you can not use a double underscore in an object name. From the file names, it is immediately clear what this archive contains: a Formex ‘F’, a Mesh ‘M’ and a TriSurface ‘T’. The file manager will normally show the file names sorted so that it is easy to see what each object consists of. It is also conceivable to delete some files from the archive, or to extract the archive, move, delete, change or rename some files, and zip the resulting files again.

The third field in the file name is the class attribute which has its data stored in the file. For attributes like ‘coords’, ‘elems’ ‘prop’, that are numpy arrays, this is evident. Attributes whose value is a string are a bit more complex: if the string is short and simple (does not contain strange control or unicode characters), it can be encoded directly in the file name as the fourth field: the Mesh ‘eltype’ field is an example. If the string is long or contains strange characters, it is stored as a charcter array (NEW: we can now store directly as text file. See below) This needs special attention on readback to allow portability over Python2 and Python3. The ‘attrib’ attribute is even more complex: it is a dict that may contain different type of objects. The ‘color’ key for example could have a string or an int or float array type. Currently we are serializing the whole ‘attrib’ in a json format string, and store the result (as a char array).

If the third component of the file name is ‘field’, it obviously stores a Field of the object. This attribute can occur multiple times and has two more components in its name: the Field’s fldtype and fldname. The contents of the file is again an array with the Field data.

The practical use is by these two function:

from pyformex.saveload import savePZF, loadPZF
savePZF(filename,**kargs)
dic = loadPZF(filename)

Though it is currently not enforced, we suggest to always use an extension ‘.pzf’ for the file name. The dictionary dic contains the pyFormex objects with their names as keys.

Data of string type can now be stored directly as a text file in the zip archive. This is especially convenient for things that could allow editing. Modern file managers allow transparent unpacking of a file in the zip archive, editing of the file and store back into the archive.

Limitations: currently, only objects of the following classes can be stored: Coords, Formex, Mesh, TriSurface, PolyLine, BezierSpline, CoordSys, Camera settings.

See also: example SaveLoad

34.1. Functions defined in module saveload

saveload.dict2Config(d)[source]

Convert a dict to a Config.

A Config is a specialized dict type that can only take some kind of values. This will raise an exception if other values are in the dict.

saveload.bytes2str(b)[source]

Convert bytes to str

saveload.readStr(filename)[source]

Read a string from a saved numpy character

The character array may be of kind ‘S’ (bytes) or ‘U’ (unicode) but always returns an object of class str.

saveload.Geometry_pzf_dict(self)[source]

Construct common part of all Geometry pzf dicts

saveload.save2zip(zipf, namedict)[source]

Save a dict to an open zip file

Parameters:namedict (dict) – Dict with objects to store. The keys should be valid Python variable names.
saveload.savePZF(filename, **kargs)[source]

Save pyFormex objects to a file in PGF format.

Parameters:
  • filename (path_like) – Name of the file on which to save the objects. It is recommended to use an extension ‘.pzf’.
  • kargs (keyword arguments) – The objects to be saved. Each object will be saved with a name equal to the keyword argument. The keyword should not end with an underscore ‘_’, nor contain a double underscore ‘__’. Keywords starting with a single underscore are reservecfor special use and should not be used for any other object.

Notes

Reserved keywords: - ‘_camera’: stores the current camerasettings - ‘_canvas’: stores the full canvas layout and camera settings

savePZF(filename, _camera=True, _canvas=True, ...)

See also example SaveLoad.

saveload.loadPolyLine(**kargs)[source]

Create PolyLine from PZF file dict

saveload.loadBezierSpline(**kargs)[source]

Create BezierSpline from PZF file dict

saveload.loadFields(obj, pzfd, fields)[source]

Load fields from PZF file on an object

saveload.loadAttrib(obj, attrib)[source]

Load attrib dict on the object

saveload.loadPZF(filename)[source]

Load pyFormex objects from a file in PGF format.

Parameters:filename (path_like) – Name of the file from which to load the objects. It is normally a file with extension ‘.pzf’.
Returns:dict – A dictionary with the objects read from the file. The keys in the dict are the object names used when creating the file.

Notes

If the returned dict contains a camera setting, the camera can be restored as follows:

if '_camera' in d:
    pf.canvas.camera.apply(d['_camera'])
    pf.canvas.update()

This will however also restore the camera aspect ratio (width/height). If the canvas where you load has another aspect ratio, the image will become distorted: a square will look like a rectangle. This may be a desired feature (like to show a very long, thin object), but in most cases it is not. Thus you may want to set the camera aspect to the same value as your canvas, e.g. before applying the loaded settings:

d = loadPZF(filename)
settings = g.get('_camera',None)
if settings:
    settings['aspect'] = pf.canvas.aspect
    pf.canvas.camera.apply(settings)
    pf.canvas.update()

Now your image will come out with the same aspect as when saving. If the canvas where you load has a larger aspect ratio, you will get extra space at the sides; if it is much smaller, you may miss some parts at the sides. Resize the canvas to view everything.

See also example SaveLoad.