Welcome to the NetCologne GmbH open source mirroring service!

This machine mirrors various open-source projects. 20 Gbit/s uplink.

If there are any issues or you want another project mirrored, please contact mirror-service -=AT=- netcologne DOT de !

Making Obnam faster

Making Obnam faster

Lars Wirzenius (liw@liw.fi)

2015-11 Debian miniconf, Cambridge

Abstract

Obnam is the backup program I have been developing for about ten years. It seems to have mostly the right feature set, and is easy to use. Unfortunately, it is only barely faster than breeding a genetically engineered leopard whose spots encode the data that gets backed up. This talk covers my efforts of speeding up Obnam so it is usable for people with real amounts of data and finite patience.

Obnam is a backup program

Glossary

Obnam is not fast: Real data (1)

Home file server to home backup server.

This is not impressive.

Helllooooo, leopard spots.

Obnam is not fast: Real data (2)

Laptop to remote backup server.

This backup is encrypted. The image is of an encrypted leopard.

Obnam is not fast: Real data (3)

Laptop to home backup server.

This is acceptable, not just tolerable. No leopard for you.

Obnam is not fast: benchmarks

High level overview of how Obnam works

For each new or changed live data file:

Backup repository structure

This is simplified, but not incorrect.

Why is Obnam slow?

Strategy

Strategy, concretely

Useful changes

Not so useful

Current results (2015-11-01)

Many files benchmark (1 million files, 1 byte each):

what initial no-op
FORMAT 6 6915 1015
FORMAT GREEN ALBATROSS 842 355

One big file (10 GB random data):

what initial
FORMAT 6 3409
FORMAT GREEN ALBATROSS 1419

All values are times in seconds.

Things I want to try next: benchmarks

Things I want to try next: changes

Someday:

How you can help

All help most welcome. Think of the genetically modified leopard kittens!

Thank you

Images: