Skip to main content

Mounting S3 bucket as a file system on Linux (Ubuntu)

In order to mount S3 bucket on Linux you need to install ssh file system (FUSE) libraries first.
Install: aptitude install build-essential libcurl4-openssl-dev libxml2-dev libfuse-dev comerr-dev libfuse2 libidn11-dev libkadm55 libkrb5-dev libldap2-dev libselinux1-dev libsepol1-dev pkg-config fuse-utils sshfs
Follow the instantiations in S3fs WIKI:
It was using http download, not SVN.
Create a mount point for the new file system (ex. mkdir /mnt/s3)
Mount the file system:
ex.: s3fs tmpname -o use_cache=/tmp -o allow_other /mnt/s3
this mount comes with option of using a tmp as a cache for S3 content, you can clean that cache as you feel fit.


You can also add your bucket to /etc/fstab

s3fs#mybucket /mnt/s3 fuse allow_other,url=https://s3.amazonaws.com 0 0

Update for mounting S3 bucket on CentOS 5.5 and other old distros.
CentOS 5.5 comes with old FUSE version 2.7.4 as latest in it`s repository.
You need to manually compile version 2.8.4 befor you can use S3fs.

Remove old FUSE
yum remove fuse fuse* fuse-devel

Install needed libraries:
yum install gcc libstdc++-devel gcc-c++ curl curl* curl-devel libxml2 libxml2* libxml2-devel openssl-devel mailcap

Download new FUSE from the Sourceforge:
wget "https://downloads.sourceforge.net/project/fuse/fuse-2.X/2.8.4/fuse-2.8.4.tar.gz?r=&ts=1299709935&use_mirror=cdnetworks-us-1"

Untar and compile:

tar -xzvf fuse-2.8.4.tar.gz
cd fuse-2.8.4
./configure --prefix=/usr

make; make install


Configure FUSE in the system:
export PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/lib64/pkgconfig/
ldconfig
modprobe fuse


Confirm that 2.8.4 is the version of FUSE displayed :
pkg-config --modversion fuse

Proceed with regular S3FS installation.

Have questions? Just contact us right away and we will be happy to assist

Forthscale is an AWS solution provider

Comments

Popular posts from this blog

How to set or disable auto-logout on bash shell

Following information is intended for bash shell only.
The system variable TMOUT can be set to specify the amount of time the user is inactive before the user is automatically logged out.
Set the following listed below in the /root/.bashrc file: TMOUT=XXX #
The system will logout the root account after one hour of inactivity. The value entered for the TMOUT= variable is in seconds. To represent 1 hour, we have to multiply as follows: 1 hour is 60 minutes. 60 minutes * 60 seconds = 3600 seconds.
After setting the parameter, the intended user will need to log out and then log in for the changes to take effect. To make modifications apply to all users on the system, specify the TMOUT variable in /etc/ profile.
To disable auto-logout simply comment out or delete the line with TMOUT value

Provided by:Forthscale systems, cloud experts

How to install PGPool II on PostgreSQL Servers in master-slave architecture + PGPoolAdmin web managment

General InformationPGPool can run on same server along with PostgreSQL DB or on stand alone server(recommended). In this article we will install PGPool on stand alone server, but the only difference is connection ports on PGPool and PostgreSQL. We will install PGPool II 3.1 on PostgreSQL 9.1.


Basic architecture: ┏───────────────┓ │                │ │    pgpool-1    │ │  pgpool server │ │                │ ┗───────────────┛ //             \\ //               \\ //                 \\ \\//                \\// \/                  \/ ┏───────────────┓           ┏───────────────┓ │                │           │                │ │    pgsql-1     │ streaming │     pgsql-2    │ │  pgsql master  │══════════>│  pgsql slave   │ │     server     │replication│     server     │ │                │           │                │ ┗───────────────┛           ┗───────────────┛


Fail cases:Slave failsIn case slave server will fail PGPool will start failover.sh script and will mark server as Down (state 3). I…

Setting Filezilla Server on Amazon EC2 instance with passive ftp

If you want to set Filezilla  ftp server to handle passive connections on your AWS EC2 instances, you should do the following.

Select non used TCP ports, for example 9024-9048 range

Configure firewalls:

In your AWS EC2 security group, allow the incoming connections on chosen ports:

tcp port 20
tcp port 21
tcp port 9024-9048

If using Windows firewall  on you instance, allow connections on same ports.

Now configure Filezilla to use specific port range on Passive connections:

Open Filezilla management console.

Got to: Edit > Settings > Passive Mode Settings

'External Server IP Address for passive mode transfers'

If you use AWS Elastic IP, enter it in "Use the following IP",
if not - use Filezilla provided web service with "Retrieve external IP address from" option.

Check 'Don't use external IP for local connections'

Check 'Use custom port range'

Enter chosen values (in our example) 9024 - 9048 for custom port range.

Powered by 123Con…