Posts from ‘Servers’

Sep
27

The Problem

After the bash exploit ‘shellshock’ was released a few days ago I’ve been going around my servers and applying the required patches, however after doing a ‘apt-get update’ on one of the web servers PHP based requests were no longer working.

Having a look in the Nginx error logs I found that the issue appeared to be at the PHP-FPM layer of the server (which I kind of expected), as it did have an update included in the bulk install and it was PHP that seemed to be broken, heres an example log:

2014/09/26 05:24:28 [crit] 26963#0: *19 connect() to unix:/var/run/php5-fpm.sock failed (13: Permission denied) while connecting to upstream, client: 46.226.191.96, server: subnet.im, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "subnet.im"
2014/09/26 05:24:29 [crit] 26963#0: *19 connect() to unix:/var/run/php5-fpm.sock failed (13: Permission denied) while connecting to upstream, client: 123.226.191.96, server: subnet.im, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "subnet.im"
2014/09/26 05:24:30 [crit] 26963#0: *19 connect() to unix:/var/run/php5-fpm.sock failed (13: Permission denied) while connecting to upstream, client: 123.226.191.96, server: subnet.im, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "subnet.im"
2014/09/26 05:24:32 [crit] 26964#0: *28 connect() to unix:/var/run/php5-fpm.sock failed (13: Permission denied) while connecting to upstream, client: 123.226.191.96, server: subnet.im, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "subnet.im"
2014/09/26 05:24:38 [crit] 26964#0: *37 connect() to unix:/var/run/php5-fpm.sock failed (13: Permission denied) while connecting to upstream, client: 123.226.191.96, server: subnet.im, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "subnet.im"

After some digging around I found that this was caused by a PHP bug fix #67060 (linky here), the bug was basically providing possible privilege escalation on the web server which they’ve fixed, however this changes some of the permissions stopping Nginx connecting to the required stocket used for PHP processing.

The Fix

Fortunately the fix is fairly simple, edit the PFP-FPM configuration.

 nano /etc/php5/fpm/pool.d/www.conf

Add in these three lines, they are probably already there and just need the comment marks removing.

listen.owner = www-data
listen.group = www-data
listen.mode = 0660

Finally re-start the PHP-FPM service and you should be back in business.

sudo service php5-fpm restart
Aug
27

yum-via-proxy-yum-package-management-mascotIntroduction.

TL,DR; – Go to Installing Squid

Yum is a great package manager for CentOS that is the secret envy of every Windows system administrator on the planet, however there will come a time when you attempt a “yum update” or “yum install tcpdump” to find out there is a problem with internet access from your server.

90% of the time you’ll probably find a network issue or someones messed up the DNS resolver configuration, however in some instances the server will legitimately have no internet access and setting up this access is either not allowed or high innocent.

Recently I worked on a server with two network connections, one to the management network and another to a VoIP signalling/media network, in this setup the default gateway was configured via the VoIP network as that’s the mission critical services, all the management elements had static routes via the management interface gateway. The problem was the VoIP network was internal and had no internet access available where as the management network did. Placing a static route for every possible Yum repository and mirror obviously isn’t an option and neither was switching around the network configuration, so here comes the Proxy.

The concept of a proxy is fairly simple, we’re going to tell Yum that all of it’s traffic should be sent to a specific IP address on a specific port, this IP address will be on a server with internet access and will have the Squid proxy installed and listening on that port for inbound connections. Assuming the access lists on the proxy are configured correctly this will then route that traffic to the internet and back on behalf of the originating server, therefore giving the illusion of internet access for Yum, simple!

 

Initialling Squid

So you need to find a server on your network that has IP connectivity to the internet and to your other server that doesn’t have internet access, this is where the proxy (Squid) will reside.

First step use Yum to install the Squid application on this server, and then ensure that it’s going to start at boot.

yum -y install squid
chkconfig squid on

Now you need to define which client IP addresses are permitted to use your proxy, in our case this range should include the IP of the client that doesn’t have internet access. So edit the squid configuration as below replacing the IP range as per your network.

nano /etc/squid/squid.conf
acl allowed_clients_acl src 192.168.0.0/24
http_access allow allowed_clients_acl

Now restart the Squid service to apply the configuration changes:

service squid restart

It’s always worth checking that Squid is actually running and listening on the correct network port using netstat

netstat -lnutp | grep 3128
tcp        0      0 0.0.0.0:3128                0.0.0.0:*                   LISTEN      20653/(squid)

Client/Yum configuration

So our Squid proxy server should be working now, the next step is to actually configure the clients to use this server. Simply in the users (in this case root) bash profile were going to specific an environment variable that yum will pick up on, so edit that profile text file:

nano /root/.bash_profile

Then just paste in this line, replacing the IP address with your Squid server (you can also use a hostname).

export http_proxy=http://192.168.204.251:3128

Bingo – Try some yum commands on the server and you should be in business!

Any problems leave a question in the comments 🙂

Dec
21

Although the Raspberry Pi comes with a HDMI port most projects are ‘head’ less (without a display), this means you spend a lot of time using either VNC or SSH to access the operating system, if it’s the latter you will normally get the most basic and boring login banner, this login banner is your MOTD (Message of the day, Linux term).

Theres a great post on the Raspberry PI forums, where someone has created the below dynamic banner for each time that you login.

rasplogin

Hopefully one day this might get included into the standard operating system, if you can’t wait like me it’s fairly simple to get it installed.

Note; See the comments for some tips on a better place to put this script

First you need to edit your profile:

sudo nano /home/pi/.bash_profile

Then just past in the code below, anywhere within that file:

let upSeconds="$(/usr/bin/cut -d. -f1 /proc/uptime)"
let secs=$((${upSeconds}%60))
let mins=$((${upSeconds}/60%60))
let hours=$((${upSeconds}/3600%24))
let days=$((${upSeconds}/86400))
UPTIME=`printf "%d days, %02dh%02dm%02ds" "$days" "$hours" "$mins" "$secs"`

# get the load averages
read one five fifteen rest < /proc/loadavg

echo "$(tput setaf 2)
.~~.   .~~.    `date +"%A, %e %B %Y, %r"`
'. \ ' ' / .'   `uname -srmo`$(tput setaf 1)
.~ .~~~..~.
: .~.'~'.~. :   Uptime.............: ${UPTIME}
~ (   ) (   ) ~  Memory.............: `cat /proc/meminfo | grep MemFree | awk {'print $2'}`kB (Free) / `cat /proc/meminfo | grep MemTotal | awk {'print $2'}`kB (Total)
( : '~'.~.'~' : ) Load Averages......: ${one}, ${five}, ${fifteen} (1, 5, 15 min)
~ .~ (   ) ~. ~  Running Processes..: `ps ax | wc -l | tr -d " "`
(  : '~' :  )   IP Addresses.......: `/sbin/ifconfig eth0 | /bin/grep "inet addr" | /usr/bin/cut -d ":" -f 2 | /usr/bin/cut -d " " -f 1` and `wget -q -O - http://icanhazip.com/ | tail`
'~ .~~~. ~'    Weather............: `curl -s "http://rss.accuweather.com/rss/liveweather_rss.asp?metric=1&locCode=EUR|UK|UK001|NAILSEA|" | sed -n '/Currently:/ s/.*: \(.*\): \([0-9]*\)\([CF]\).*/\2°\3, \1/p'`
'~'
$(tput sgr0)"

Source: http://www.raspberrypi.org/phpBB3/viewtopic.php?t=23440

Dec
03

As a system admin or even just a Linux enthusiast you will most likely find yourself dabbling within log files from time to time, while troubleshooting it can often help to watch the logs come into your physical files as they occur, instead of searching for them after the event.

One example might be the apache error.log for a web server, if you navigate to the directory you can run the following command to get a live output of the file.

tail -f ./error.log

Then replicate your problem and have a look at what log’s come through.
Some log files like the one above can be quite busy, especially something like a web access log on your live web server, so it’s also a good idea to filter your output using grep.

In the example below, I want to see all errors, but only for the web client having the errors (my PC)

tail -f ./error.log | grep 192.168.0.123

As you can see it’s just a simple case of piping the output into the grep command.
Once your done with either command just hit ctrl + c to cancel out of the live trace.

>Ctrl^C
Jun
03

I have been doing quite a bit of development work recently on CentOS and Debian systems, I am sure like me you will find yourself endlessly typing the same commands over and over again in your SSH terminal. Sure you can just go up through your BASH history or copy and paste the command, however it’s a bit messy.

Say hello to the great command Alias, this allows you to define a word to use in your bash prompt that will run a command, a kind of macro/shortcut for the world of bash.

Example

Here is a simple example; while developing PHP I like to get a quick view of my error.log on the live webserver so I can proactively go through and fix any problems, the below command accomplishes this and also filters to show just logs form my IP.

tail -f /home/beta/log/error.log | grep 1.2.2.3
2012/05/31 21:32:06 [error] 9777#0: *1055599 connect() failed (111: Connection refused) while connecting to upstream, client: 1.2.2.3, server: beta.route.im, request: "GET /robots.txt HTTP/1.1", upstream: "http://127.0.0.1:8000/robots.txt", host: "beta.route.im"
2012/06/01 05:53:37 [error] 9777#0: *1059307 connect() failed (111: Connection refused) while connecting to upstream, client: 1.2.2.3, server: beta.route.im, request: "GET /robots.txt HTTP/1.1", upstream: "http://127.0.0.1:8000/robots.txt", host: "beta.route.im"

However I would much prefer if I could just type a single word to see the same output, lets use betalog, here is the alias command:

alias betalog="tail -f /home/beta/log/error.log | grep 1.2.2.3"

Great, so now look at my output 🙂

betalog
2012/05/31 21:32:06 [error] 9777#0: *1055599 connect() failed (111: Connection refused) while connecting to upstream, client: 1.2.2.3, server: beta.route.im, request: "GET /robots.txt HTTP/1.1", upstream: "http://127.0.0.1:8000/robots.txt", host: "beta.route.im"
2012/06/01 05:53:37 [error] 9777#0: *1059307 connect() failed (111: Connection refused) while connecting to upstream, client: 1.2.2.3, server: beta.route.im, request: "GET /robots.txt HTTP/1.1", upstream: "http://127.0.0.1:8000/robots.txt", host: "beta.route.im"

One final thing to remember

Alias commands only work in the current bash session, therefore when you close your SSH window they will be lost, fortunately it’s fairly easy to save them for use at a later date you just need to save it in a user profile.

Normally this will be in your home directory called ‘.profile’, on my debian system that actually referances another file in my home directory ‘.bashrc’.

cd ~
nano .bashrc

Finally these alias commands are stored in the user profile, which if you don’t already know means they will be only available for the user with that profile in their home directory.

Hope that helped, any questions please leave in the comments below :), I’m going back to coding some more!

Our Sponsors