Script to Clean Up F-Spot Database

2009-10-14T00:30:12-07:00

One of the more popular photo management applications for Linux is f-spot, but unfortunately it has a rather glaring bug. It uses a sqlite3 database internally to track which pictures you’ve imported into it, what tags they have, etc. However, if you delete, move or rename any of the files that f-spot is tracking, the next time you browse to that photo within f-spot, it will crash the program. It’s annoying, and there’s no particularly easy way to deal with it…until now.

The script below simply iterates through all of the photos that are in f-spot’s database, and checks to see if those photos exist on your hard drive. If you run it in demo-mode, it will show you which files look problematic, and if you run it in normal mode, it will delete those database entries so that your database is cleaned up (but not before backing up your database, just in case).

In my very limited testing, it works very well, any additional feedback or bug reporting is more than welcome.

#!/bin/bash

# A script to find missing files in the f-spot database, and then delete them.
# At present these files crash f-spot. It's frustrating as all hell.

echo "Welcome to the f-spot database cleaner. All the usual disclaimers apply, as you might imagine."
echo "What would you like to do: " 
echo "  1) Run in demo-mode " 
echo "  2) Clean up your f-spot database" 
echo "  3) Quit" 
read -p "Your choice: " choice

case $choice in 
    1) demomode="true";;
    2) demomode="false";;
    3) exit 0;;
esac

# With that beginning stuff out of the way, let us do some functions
# First, a function to gather the database contents and to print out the ones that are orphans

function findAndFixOrphans {
    # find our db, and set a var. Checking for XDG path first, since it's the more recent location of the db
    if [ -f $XDG_CONFIG_DIR/f-spot/photos.db ] #checks if the $XDG_CONFIG_DIR variable is in use
    then
        DBPATH=$XDG_CONFIG_DIR/f-spot/photos.db
    elif [ -f $HOME/.config/f-spot/photos.db ] #uses the default $XDG location, if that's being used.
    then
        DBPATH=$HOME/.config/f-spot/photos.db
    elif [ -f $HOME/.gnome2/f-spot/photos.db ] #uses the old location of the DB, if the former aren't in use.
    then
        DBPATH=$HOME/.gnome2/f-spot/photos.db
    else
        echo "Error: Could not find database. Damn." 
        exit 1
    fi

    # Select the filenames, and put them in a variable.
    filenames=$(sqlite3 $DBPATH "SELECT URI FROM PHOTOS")
    filenames_versions=$(sqlite3 $DBPATH "SELECT URI FROM PHOTO_VERSIONS")

    # Chomp off the first instance of file://, and replace the rest with newlines.
    filenames=$(echo $filenames | sed 's/file:\/\///' | sed 's/file:\/\//\n/g'  )
    filenames_versions=$(echo $filenames_versions | sed 's/file:\/\///' | sed 's/file:\/\//\n/g' )

    if [ $demomode == "true" ]
    then            
        while read -r line
        do
            # Decode the filename
            decodedLine=$(echo -e "${line//\%/\\x}")
            if [ ! -f  "$decodedLine" ] 
            then
                # If the file doesn't exist, we output the filename, if in demomode, or we fix it if we are not in demomode.
                echo  "Errant record found in the photos table: $decodedLine"
            fi
        done <<< "$filenames"

        # We do the same for the photo_versions table
        while read -r line
        do
            # Decode filename
            decodedLine=$(echo -e "${line//\%/\\x}")
            if [ ! -f "$decodedLine" ]
            then
                # If the file doesn't exist, we output the filename
                echo "Errant record found in the photo_versions table: $decodedLine"
            fi
        done <<< "$filenames_versions"

    else
        # We backup the database, and make the correction
        cp $DBPATH $DBPATH.`date -I`.bak
        if [ $? -eq 0 ]
        then
            #The backup worked, tell the user.
            echo "Your database has been backed up to $DBPATH.`date -I`.bak"
        else
            echo "Error backing up your database."
            exit 3
        fi

        # First we do the photos table
        while read -r line
        do
            # Decode the filename
            decodedLine=$(echo -e "${line//\%/\\x}")

            if [ ! -f "$decodedLine" ]
            then
                # Do some sql here.
                foo="file://${line}"
                echo -n "Deleting URI $line from the database table photos..."
                sqlite3 $DBPATH "DELETE FROM PHOTOS WHERE uri = '$foo'"
                echo "done."
            fi
        done <<< "$filenames"

        # Then we do the photo_versions table
        while read -r line
        do
            # Decode the filename
            decodedLine=$(echo -e "${line//\%/\\x}")

            if [ ! -f "$decodedLine" ]
            then
                #Do some sql stuff
                foo="file://${line}"
                echo -n "Deleting URI $line from the database table photo_versions..."
                sqlite3 $DBPATH "DELETE FROM PHOTO_VERSIONS WHERE uri = '$foo'"
                echo "done."
            fi
        done <<< "$filenames_versions"

    fi

}

if [ "$demomode" == "true" ]
then
    echo "Great. Proceeding in demomode."

    findAndFixOrphans

    echo "Demomode successfully finished. Exiting."
    exit 0;
elif [ "$demomode" == "false" ]
then
    echo "Great. Cleaning up your database."

    findAndFixOrphans

    echo "Database cleaned successfully."
    exit 0;
else
    echo "Something strange happened. See the script for details. Exiting."
    exit 2;
fi

Delete Original Images from F-Spot and Rotate Using Exif Information

2008-09-15T23:33:48-07:00

Over the past couple weeks, my girlfriend and I spent a lot of time working on the photos from our Peru trip. Since we used F-Spot to do our photo editing, when we were done we had a couple of problems when it came to transferring the photos back to her computer which is running Vista.

The first problem was that when we rotated images with F-Spot, it simply changed the exif information for the photo, and didn’t change the pixels of the photo themselves. This was fine when viewed in F-Spot because it is aware of exif information, and displays the photos correctly. However, when we transferred the files to her computer, we discovered that Vista does not take exif information into account on the OS level, nor does Picassa. As a result, we needed to somehow rotate the images that had exif information indicating a non-normal rotation.

The second problem we encountered is that in F-Spot when you edit a photo, it creates a second file with the edited photo, and leaves the original unchanged. So, if you edit file dsc00343.jpg, you get a second photo called dsc00343 (Modified).jpg. This is OK when in F-Spot, however, when we went to her computer, it was very hard for her to have ONLY the modified version of those photos, and to delete the originals (since the edited versions are better than the originals).

To solve the first problem, I used a couple of tricks. The first thing I did was to make a copy of the photos in case all went south.

mkdir backupPhotos
cp *.jpg *.jpeg backupPhotos
cd backupPhotos

Once that is done, we can begin rotating images. For this, we will need the jhead program.

sudo aptitude install jhead

Once that’s installed, we rotate:

jhead -autorot  *.jpg *.jpeg

This will rotate all of the files that have unusual information in the exif orientation field.

Problem number one solved.

For problem two, we will need to isolate all of the photos that have been modified, and delete the originals. To do this, we will capitalize on the fact that the renamed images use the original pictures name in their name.

To begin with, we create a list of the photos that have been modified:

ls *.jpg *.jpeg | grep -i modified > modifiedImages.txt

Using awk, we can separate out the original name of the photos. The following commands will convert the ‘)’ to ‘(‘ and will use the two ‘(‘ as a delimiters, returning the name of the file as the first field, and the .jpg or .jpeg as the third field. After that, it will remove any spaces from the file name, and will create a new file with a list of the modified pictures. It sounds complicated, but the final result should work:

tr ')' '(' &lt; modifiedImages.txt > modifiedImages2.txt
cat modifiedImages2.txt | awk -F'(' '{print $1,$3;}' | sed 's/  //g' &gt; modifiedImages3.txt

You should now have a file called modifiedImages3.txt that contains the name of all of the original photos. To delete the pictures in this list from the collection - permanently - run:

rm `cat modifiedImages3.txt`

You should now be able to transfer this entire directory of photos to another computer without rotation issues or duplicated photos.

Michael Jay Lissner

Script to Clean Up F-Spot Database

Delete Original Images from F-Spot and Rotate Using Exif Information