Graphite and collectd

So …  Recently and due to a need from the company I work for, I had to search for a tool that could gather all the metrics from several systems. Mainly the focus is about centralization.

Initially I had the idea of sar, but most of us know, sar is single system only, meaning that I would have to run sar in multiple systems and then grab the output files from each system and treat the data to plot the graph.
Since we are converging into a centralized form I notice one tool that is being used by the development team to gather metrics from their applications, the tool is Graphite. This tool is indeed pretty good, it allow us to have a central system to look at the data. It allows us to have a centralized point and also merge graphs.

In our particular case, we ended up but having Graphite and an extra tool for metrics gathering .
What is collectd? It is our core so to speak, collectd is responsible for the collection of the metrics where the agent is running. This agent has an enormous amount of plugins that allow you do gather statistics on almost everything. Check here for the list of plugins . Also, you can have relays agents in case you actually need then or

Collect has the ability to forward all the data to graphite. At the present time, the plugin is write_graphite, so whatever you configure, all your metrics will be sent to graphite and you can easily look at them in a graphical format.

tmux and screen

So which one do you prefer?

Personally I love tmux mainly for these 2 big reasons:

  1. ability to split windows vertical or horizontal, useful when in need to debug a program or script.
  2. configuring it according to my taste is a lot easier than screen.

So inline will be example for the config for both software’s:

Screen – for most of my case scenarios this is more than enough, but if you want to play around with the settings and configs then you will have a hard time digging in screen manual. Although I search online before I was happy, I ended up by having this config for my screen:

~/.screenrc
altscreen on
startup_message off
caption always '%{= dr} %H %{G}| %{w}%l %{G}|%=%?%{d}%-w%?%{r}(%{b}%n %t%? {%u} %?%{r})%{d}%?%+w%?%=%{G}| %{y}%M %d %c:%s '
defscrollback 5000

Tmux - tmux does what screen does but it’s configuration is a lot easier to perform. As you can see by my config, it is human friendly unlike screen. For this config to work as expected, you should have an alias for tmux as ‘tmux -2′.

~/.tmux.conf
set -g default-terminal "screen-256color"
set -g history-limit 10000

############################################################################
# Status Bar
############################################################################
set-option -g status-utf8 on
set-option -g status-justify centre
set-option -g status-bg black
set-option -g status-fg cyan
set-option -g status-interval 3
set-option -g status-left-length 50
set-option -g status-left ‘#[fg=red]» #[fg=blue]#h# #[fg=colour128]#(uptime |cut -d ‘,’ -f 3,4,5)#[default]‘
set-option -g status-right ‘#[fg=red] #[fg=cyan] »» #[fg=blue]###S #[fg=colour184]%R %d-%m-%y#2#[default]‘

set-option -g visual-activity on
set-option -g set-titles on
set-option -g set-titles-string ‘#H’

set-window-option -g window-status-current-bg red
setw -g mode-keys vi

bind-key : command-prompt
bind-key r refresh-client
bind-key L clear-history
bind-key v split-window -h
bind-key h split-window -v

The new generation of Unix System Administrators …

I am currently a System Administrator and I’ve been working in it for 12 years now. During the time that I started working, all the systems administrators shared a common characteristic, we were all curious about system administration. There could be already a way of performing a task but we would look for ways to improve this.

Now, here comes the “funny” part … nowadays, Unix System Administrator are simply lazy1 with lack of commitment and no ambitions.
Sometime I wonder if they are in IT due to this way of thinking: “IT is well paid”. I wouldn’t be surprise that this is the case for a few of people as it has already happened in a not so far away past.

Some example are as follows:
“Tell me where I can download “, “How can I add a username?”, “Tell me how to update the this system!”, “Where can I find the log files?” and very similar things to these.

Honestly, most of the times I would like to be able to just write RTFM … this is something that it doesn matter in which area of IT you are in, you NEED to read the manual of the command or of the operating system that you work with.

Long time ago, when I was still studying, one of my teachers said this “A system administrator is just as a medical doctor, it has to be always up to date with the current situation and are advancements!”

Vim for perl and python development

Long time since I’ve been here but it happens …

Due to my employment changes and new areas of expertise, I found myself with the need to learn Perl and Python.

I am currently searching for plugins to work with Python for syntax checking since there is already a plugin for Perl that does exactly this.

So let’s start with Perl. As many of you know, Perl is very well accepted across sysadmins worldwide and has a very big library to back it up. So, if you are developing in perl and you are using Vim as your “IDE” it is a good idea to install the plugin.

From this point on whenever you create a new file with .pl you will be presented with a template. This template can be changed to match your needs. But so far, what I found more interesting on this plugin is when you sequence \rpc, this will evaluate your code according to the book Perl Best Practices. It will point your errors and it will also indicate where in book you can find the information why it is incorrect. As an example:

  1 update_settings.pl|56 col 1| Always unpack @_ first.  See page 178 of PBP  (Severity: 4)$
  2 update_settings.pl|72 col 48| “die” used instead of “croak”.  See page 283 of PBP  (Severity: 3)$

Give it a try, it is worth fit, specially if you are learning how to code Perl.

As for Python, I found that it is pretty annoying the indent system where it has to match 4 spaces. But it is as simple as using some options in Vim to make it work for you. Configure your .vimrc to look like this:

set expandtab

set tabstop=4

set softtabstop=4

set shiftwidth=4

set autoindent

set number

set list

The most important are expandtab that converts your tab to spaces and the tabstop, python uses 4 spaces for ident.

Enjoy your coding!! :)

Virtual test environment for Cluster 3.2

It’s been a while, yes as you can see I’m not a full tine blogger. :)
Anyway, due to the nature of my work I need to test some stuff first before applying them live. As such it’s a bit expensive to have several servers just for that.
It’s also good if we want to test something new like really brand new, a new feature, a patch that gets more things done and so on or even to get ready for a certification exam.
So this post will be mostly about the initial setup using the following Software

      1. Host: My laptop with Fedora 13 x86_64

 

      2. Virtualization Software: Oracle Virtual Box 3.2.8

 

      3. Arquichiture: 2-node cluster

 

      4. Software: Oracle Cluster 3.2u3

 

    5. Operating System: Oracle Solaris 10/09 x86

Initial difficulties that I had:

      1. Where would I put the /globaldevices filesystem?

 

      2. How would I configure the interconnect interface?

 

    3. What would be the quorum device?

And the answer that I found was as follows:

      1. Searching a bit I got the following link for the wikis.sun.com. In here it states that we can use globaldevices with ZFS.

wikis.sun.com

    2. Another thing I had doubt with was regarding the interconnect. At least the Virtual Box has a internal network interface and a host only interface. Since in the case of the interconnect they have to be completely isoladed from any traffic I had to create two internal networks. The command used was:

VBoxManage –modifyvm –intnet1 interconnect1
VBoxManage –modifyvm –intnet1 interconnect1
VBoxManage –modifyvm –intnet2 interconnect2
VBoxManage –modifyvm –intnet2 interconnect2

Afterwards in the GUI of VBOX just needed to add the interfaces adapter 2 and 3 to match the interconnects.
3. As for the Quorum device … well in here I had 3 choices:

        3.1 Quorum disk device

 

        3.2 Nas Server – Requires another vm

 

      3.3 Quorum server

I ended up going with Quorum disk device thanks to Virtual Box being able to share the same disk across Virtual Machines.
The requisites to be able to do this are:

      1. The disk image file

MUST NOT

      be dinamycally.

 

      2. Before applying the disks to a Virtual Machine we need to execute the following command:

VBoxManage modifyhd --type shareable

After applying the change you can had normally to the Virtual Machine. In my case I added a SAS controller to each Virtual Machine and added the shared devices under that SAS controller.

Now …
From this point forward is quite simple. You just need to install each system accordingly.
So …

      1. Install Solaris in each node.

 

      2. Permit ssh login from one node to the other with ssh exchange keys

        2.1. Edit PermitRootLogin=no to yes in /etc/ssh/sshd_config

 

        2.2. Generate the ssh keys for user root:

 

        solaris10-1# ssh-keygen -t rsa

 

        2.3. Copy the .pub file to second node

 

        solaris10-1# scp id_rsa.pub root@solaris10-2:/.ssh/authorized_keys

 

        2.4. Repeat steps 1 to 3 from the second node

3. Install Cluster Software in each node and perform initial configuration.
NOTE: When performing the configuration it will ask for the interconnect devices. Don’t forget to specify the interconnects as the previous adapters that are under the Virtual Box internal network.
4. After all done, it’s time to specify the quorum device.

        4.1 Properly idenfify the shared devices. In my case they were under c2*

 

        4.2 Label each disk to be recognized by Solaris

 

        4.3 Create partitions as you wish.

 

      4.4 Execute in both nodes cldev populate; cldev show

From this point forth you are ready to toy a bit with your two node cluster. :)
By the way, you will have several messages regarding the system being unable to write the the keys to the quorum, at this point I haven’t looked into it properly. I will update when I have the time to take a depth look into it.

Fedora 13 and Sun Java installation

Back so soon, yeah I know …

Another issue that I came across regarding Java. In my day to day work my major application is dependent on Java and it cannot be IcedTea or the java-gcj version. It really has to be Sun version.

At first I was thinking it would be a nightmare, but after searching a bit I found what I was looking for. In a way of compiling the big information, here it goes what I’ve done to install it and configure it properly in my system

1. step -> Download java JDK rpm file.

http://java.sun.com/javase/downloads/index.jsp

at the time of this writing: jdk-6u20-linux-x64-rpm.bin

2. step – Install the rpm file, it’s going to install several rpm actually
# chmod +x jdk-6u20-linux-x64-rpm.bin

The java will be installed at: /usr/java
lrwxrwxrwx. 1 root root 16 May 6 15:32 default -> /usr/java/latest
drwxr-xr-x. 9 root root 4096 May 6 15:32 jdk1.6.0_20
lrwxrwxrwx. 1 root root 21 May 6 15:32 latest -> /usr/java/jdk1.6.0_20

After this you will need to configure the alternatives to recognize the new java version installed. I’ve saved a file with all that is needed, or at least for all that I need.

/usr/sbin/alternatives –install /usr/bin/java java /usr/java/latest/bin/java 20000 \
–slave /usr/bin/javaws javaws /usr/java/latest/bin/javaws \
–slave /usr/bin/keytool keytool /usr/java/latest/bin/keytool \
–slave /usr/bin/orbd orbd /usr/java/latest/bin/orbd \
–slave /usr/bin/rmid rmid /usr/java/latest/bin/rmid \
–slave /usr/bin/rmiregistry rmiregistry /usr/java/latest/bin/rmiregistry \
–slave /usr/bin/servertool servertool /usr/java/latest/bin/servertool \
–slave /usr/bin/tnameserv tnameserv /usr/java/latest/bin/tnameserv \
–slave /usr/bin/pack200 pack200 /usr/java/latest/bin/pack200 \
–slave /usr/lib/jvm/jre jre /usr/java/latest/jre \
–slave /usr/bin/unpack200 unpack200 /usr/java/latest/bin/unpack200

You can now choose what version of java you want to use by issuing the command:
# alternatives –config java

There are 3 programs which provide ‘java’.

Selection Command
———————————————–
1 /usr/lib/jvm/jre-1.6.0-openjdk.x86_64/bin/java
2 /usr/lib/jvm/jre-1.5.0-gcj/bin/java
*+ 3 /usr/java/latest/bin/java

Hope it helps anyone out there.

My problem installing Android with Ecplise in Fedora

Hello all,

Long time since I’ve posted anything at all.
Well, I got curious about developing for a mobile platform, so I decided to try it out. Guess what? What the hell did I got myself into. :D

Not sure if everyone got this problem but I indeed had it and so I had to perform some steps. Before anyone says something here is my initial Fedora installation as anaconda-ks.cfg file:

@admin-tools
@base
@core
@editors
@fonts
@gnome-desktop
@games
@graphical-internet
@graphics
@hardware-support
@input-methods
@java
@office
@online-docs
@printing
@sound-and-video
@text-internet
@base-x
xfsprogs
mtools
gpgme
gnupg2
openoffice.org-opensymbol-fonts
gvfs-obexftp
dcraw
hdparm
m17n-db-tamil
m17n-db-gujarati
m17n-db-kannada
m17n-db-hindi
gok
m17n-db-oriya
m17n-db-bengali
m17n-contrib-sinhala
m17n-db-assamese
m17n-db-punjabi
iok
m17n-db-telugu
m17n-db-malayalam
jack-audio-connection-kit
gdm

Beside this I installed the Eclipse that comes with Fedora 13. And let me tell you, forget it, it simply doesn’t work when we’re following the steps at http://developer.android.com/sdk/index.html. Since I’ve done this some months back I don’t have the exact error messages but basically it has to do about version.

So I had to remove Fedora Eclipse and download the Eclipse from http://www.eclipse.org.
Regarding the android part and since I installed the Desktop version of Fedora and not full I had to install a few packages that I did with the following steps, based on try and error of course.

yum whatprovides ld-linux.so.2
yum install glibc-2.11.1-6.i686
yum whatprovides libX11.so.6
yum install libX11-1.3-1.fc12.i686
yum whatprovides libstfc++-so.6
yum whatprovides libstdc++.so.6
yum install libstdc++-4.4.2-7.fc12.i686
yum whatprovides libstdc++.so.6
yum install libstdc++-4.4.3-4.fc12.i686
yum whatprovides libncurses.so.5
yum install ncurses-libs-5.7-3.20090207.fc12.i686
yum install libXrandr-1.3.0-5.fc12.i686 <—– SDL initialization failed.
yum install -y alsa-lib-1.0.23-1.fc12.i686
yum install -y libao-0.8.8-7.fc12.i686
yum install zlib-1.2.3-23.fc12.i686 <—– eclipse related

After this it was as simple as following the information at the android site.

Hope it helps anyone out there.