Thursday, April 07, 2011

Mounting S3 bucket as a file system on Linux (Ubuntu)

In order to mount S3 bucket on Linux you need to install ssh file system (FUSE) libraries first.
Install: aptitude install build-essential libcurl4-openssl-dev libxml2-dev libfuse-dev comerr-dev libfuse2 libidn11-dev libkadm55 libkrb5-dev libldap2-dev libselinux1-dev libsepol1-dev pkg-config fuse-utils sshfs
Follow the instantiations in S3fs WIKI:
It was using http download, not SVN.
Create a mount point for the new file system (ex. mkdir /mnt/s3)
Mount the file system:
ex.: s3fs tmpname -o use_cache=/tmp -o allow_other /mnt/s3
this mount comes with option of using a tmp as a cache for S3 content, you can clean that cache as you feel fit.


You can also add your bucket to /etc/fstab

s3fs#mybucket /mnt/s3 fuse allow_other,url=https://s3.amazonaws.com 0 0

Update for mounting S3 bucket on CentOS 5.5 and other old distros.
CentOS 5.5 comes with old FUSE version 2.7.4 as latest in it`s repository.
You need to manually compile version 2.8.4 befor you can use S3fs.

Remove old FUSE
yum remove fuse fuse* fuse-devel

Install needed libraries:
yum install gcc libstdc++-devel gcc-c++ curl curl* curl-devel libxml2 libxml2* libxml2-devel openssl-devel mailcap

Download new FUSE from the Sourceforge:
wget "https://downloads.sourceforge.net/project/fuse/fuse-2.X/2.8.4/fuse-2.8.4.tar.gz?r=&ts=1299709935&use_mirror=cdnetworks-us-1"

Untar and compile:

tar -xzvf fuse-2.8.4.tar.gz
cd fuse-2.8.4
./configure --prefix=/usr

make; make install


Configure FUSE in the system:
export PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/lib64/pkgconfig/
ldconfig
modprobe fuse


Confirm that 2.8.4 is the version of FUSE displayed :
pkg-config --modversion fuse

Proceed with regular S3FS installation.

Have questions? Just contact us right away and we will be happy to assist

Forthscale is an AWS solution provider

No comments:

solving error: Your current user or role does not have access to Kubernetes objects on this EKS cluster.

Trying to access EKS cluster with kubectl you might get an error similar to: Your current user or role does not have access to Kubernetes ob...