The first time I tried I got the error: "Error while loading shared libraries: libnotes.so: cannot open shared object file: No such file or directory."
So I created a symbolic link the library in /usr/lib with the ln command to fix this problem. Then I found another 2 libraries were missing and after I created their links, it worked!

The command ln -s is used to make symbolic links instead of hard links

Run the following commands as root to create symbolic  link the libraries:

>ln -s /opt/ibm/lotus/notes/latest/linux/libnotes.so /usr/lib/libnotes.so
>ln -s /opt/ibm/lotus/notes/latest/linux/libndgts.so /usr/lib/libndgts.so
>ln -s /opt/ibm/lotus/notes/latest/linux/libxmlproc.so /usr/lib/libxmlproc.so

Now you will be able to run the Domino programs in a Linux terminal window.
First change to the Domino binary directory, if you are at the root directory then:
cd /opt/ibm/lotus/notes/latest/linux

Next run compact
notes@tribbles:/opt/ibm/lotus/notes/latest/linux>./compact -c [/local/notesdata/filename.nsf]

If you want to clean up and really reduce the sizes of the databases then run compact with the following command after you have added the Notes.ini entry to use the ODS51 format.
Create_R85_Databases=1

./Compact -c -ZU -n -v
-c        Copy style compact, used to put the database into new ODS format.
-ZU        Converts document attachments using compression type Huffman to LZ1.
-n        Compress design
-v        Compress data

This will set the database properties before compacting the database, so you get an additional reduction in size without having to manually set the properties.