For SysAdmins and Passionates

TuxByte.com


How to configure Name-based VHosts on Apache

Hi,

Today I will show you how to configure Name-based VHosts(the default option /etc/apache2/ports.conf–NameVirtualHost *:80) in Apache http server, in this way you can have different domains been served by the same HTTP Server(Apache in this case).

I assume that you already have installed Apache and all the stuff required to, deploy your application(lamp stack, for example…).

To have multiple sites running in the same Apache server, you have to focus in two locations(assuming that you haven’t touched in /etc/apache2/ports.conf):

/etc/apache2/sites-available – Where you have all the config’s done

/etc/apache2/sites-enable – Where apache looks for config’s, this folder have, symbolic links only, for the sites-available folder above.

 

In /etc/apache2/sites-available

Inside you will find at least two default config for your site(default, default-ssl).

Like the names indicate, one is for normal http requests on port 80, the other is for https on port 443.

 

Now lets define our Multi-site Development environment(using named-based vhosts) for internal users, for example:

Lets imagine that our domain is mysite.test.And we want to Setup a domain for Paul develop her things…

we can make a copy of the default vhost file and change the name.

#cp default nameofdeveloper; Or use another suggestive name…we can use a domain name(i.e docs.mysite.test).

#vi paul;  And  change what you want(Paul is the name of the developer).

<VirtualHost *:80>

   DocumentRoot /home/paul/dev_site      #the folder where pauls site is located

   ServerName  paul.mysite.test                    #the name for what the apache will recognise this vhost

   ServerAlias  www.paul.mysite.test  foo.paul.mysite.test  *.paul.mysite.test #i.e another subdomains for my site
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory /home/paul/dev_site/>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
</Directory>

ScriptAlias /cgi-bin/ /home/paul/dev_site/cgi-bin/         #the cgi programs folder
<Directory “/home/paul/dev_site/cgi-bin/”>
AllowOverride None
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>

ErrorLog /home/paul/dev_site /log/error.log          #where the error log’s will be stored.

# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn

#CustomLog ${APACHE_LOG_DIR}/access.log combined
SetEnvIf Request_URI “^/check\.txt$” dontlog
CustomLog /var/log/apache2/access.log combined env=!dontlog
</VirtualHost>

Now save the file, and jump to sites-enable folder.

 

In /etc/apache2/sites-enable

In sites-enable folder create a symbolic link for  sites-available config file(in this case paul). Apache will look in sites-enable for config’s.

#ln -s ../sites-available/paul

We can use a tool to this automatically:

a2ensite – apache2 enable site

a2dissite – apache2 disable site

, to create or remove the symbolic links

root@irony:~# a2ensite paul

Site paul installed; run /etc/init.d/apache2 reload to enable.

or to disable:

root@irony:~# a2dissite paul

Site paul unistalled; run /etc/init.d/apache2 reload to enable.

 

Reload the apache configuration(to load the new vhosts)

#/etc/init.d/apache2 reload , or  #service apache2 reload

Now you can test it, in your browser, with the url http://paul.mysite.test

This vhost will use all the site that paul have in their home directory(/home/paul/dev_site), and will not mess with others, that can be already configured.

 

Its a 3 steps work, sites-available,sites-enable,reload!

 

Related Documentation:

http://debian-handbook.info/browse/squeeze/sect.http-web-server.html

http://www.debianhelp.co.uk/virtualhosts.htm

http://www.debian-administration.org/articles/412

 

If you don’t understand something, please feel free to comment bellow.

Enjoy ;)

Dummy test with sdcc for the great STM8 MCU

For a long time now, I which to have free tools to use with this great MCU…

STMicroelectronics, only have Windoze tools, so I run a batch of investigations(like a lot of guys out there) on how to build applications for this MCU…

For now, there are some projects that are developing tools for this great MCU, the most complete until now is the sdcc.

It includes :

—————————————-

sdcpp – Preprocessor

sdcc – Compiler

sdasstm8 – Assembler

sdld – Linker

sstm8 – ucsim, it’s an emulator based on 8051, but this version is for stm8 :P

——————————————

I downloaded the snapshot version 8889 from 2013/10/26 of SDCC, from here.

After that, extracted it, and jumped to the bin directory inside…

I created a dummy c file, main.c, (based on a article I found) to test.

———————————————–

/**main.c
Sample STM8 application.
SDCC for STM8 compiler.
*/
volatile int my_var =9;   // .data section
void main(void){
    do{
        my_var++;   //load, increment, store
    }while(1);  //never return from main()
}

————————————————

After that I tried to make all the steps of compiling .

sdcc main.c -mstm8

I ended with 8 files generated:

main.asm, main.ihx, main.lk, main.lst, main.map, main.rel, main.rst, main.sym

for now, everything goes fine…

to compile and link in separated files you can follow this example…

sdcc -c foo1.c -mstm8

sdcc -c foo2.c -mstm8

sdcc foo_main.c foo1.rel foo2.rel

I never used the ucsim emulator, and I don’t know exactly on how to simulate code there, but I am investigating…

For that I started the ucSYM stm8(sstm8) emulator…

./sstm8 main.ihx
uCsim 0.5.4, Copyright (C) 1997 Daniel Drotos, Talker Bt.
uCsim comes with ABSOLUTELY NO WARRANTY; for details type `show w’.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c’ for details.
172 words read from main.ihx <—————————————-the program memory have been readed…
******************** leave the RAM dirty now
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0×0000   0 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0×8083 ae 00 00       ldw   X,#0×0000 <————  for what I understand it initializes the 16 bits register X with 0,in next instruction
0> next
V-IHINZC  Flags= 0×02   2 .  A= 0×00   0 .
0-000010  X= 0×0000   0 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0×8086 27 07          jreq  0x808f
0> next
V-IHINZC  Flags= 0×02   2 .  A= 0×00   0 .
0-000010  X= 0×0000   0 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x808f ae 00 02       ldw   X,#0×0002 <—- I don’t understand why in the next instruction we will  load 2 to register X ??????
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0×0002   2 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0×8092 27 09          jreq  0x809d 
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0×0002   2 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0×8094 d6 80 a9       ld    A,(0x80a9,X) <————-we will load in next instruction, 9(from 0x80ab) to A?? my_var=9?
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×09   9 .
0-000000  X= 0×0002   2 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0×8097 d7 00 00       ld    (0×0000,X),A               <————— why???????????god…
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×09   9 .
0-000000  X= 0×0002   2 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x809a 5a             decw  X                                     <————why?????
0> next
V-IHINZC  Flags= 0×80 128 .  A= 0×09   9 .
1-000000  X= 0×0001   1 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x809b 26 f7          jrne  0×8094
0> state
CPU state= OK PC= 0x00809b XTAL= 8e+06
Total time since last reset= 8.75e-07 sec (7 clks)
Time in isr = 0 sec (0 clks)   0%
Time in idle= 0 sec (0 clks)   0%
Max value of stack pointer= 0×000000, avg= 0×000000
0> next
V-IHINZC  Flags= 0×80 128 .  A= 0×09   9 .
1-000000  X= 0×0001   1 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0×8094 d6 80 a9       ld    A,(0x80a9,X)
0> next
V-IHINZC  Flags= 0×82 130 .  A= 0×00   0 .
1-000010  X= 0×0001   1 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0×8097 d7 00 00       ld    (0×0000,X),A
0> next    
V-IHINZC  Flags= 0×82 130 .  A= 0×00   0 .
1-000010  X= 0×0001   1 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x809a 5a             decw  X
0> next
V-IHINZC  Flags= 0×82 130 .  A= 0×00   0 .
1-000010  X= 0×0000   0 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x809b 26 f7          jrne  0×8094
0> next
V-IHINZC  Flags= 0×82 130 .  A= 0×00   0 .
1-000010  X= 0×0000   0 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x809d cc 80 80       jp    0×8080
0> next
V-IHINZC  Flags= 0×82 130 .  A= 0×00   0 .
1-000010  X= 0×0000   0 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0×8080 cc 80 a0       jp    0x80a0
0> next
V-IHINZC  Flags= 0×82 130 .  A= 0×00   0 .
1-000010  X= 0×0000   0 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a0 ce 00 01       ldw   X,0×0001 <——-the real program start here…i don’t  know yet the asm…but seems too unoptimized….but I
0> next                                                              DONT know it…
V-IHINZC  Flags= 0×80 128 .  A= 0×00   0 .
1-000000  X= 0×0009   9 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a3 5c             incw  X <——————— the first increment of our var in C code above, my_var…
0> state
CPU state= OK PC= 0x0080a3 XTAL= 8e+06
Total time since last reset= 1.875e-06 sec (15 clks)
Time in isr = 0 sec (0 clks)   0%
Time in idle= 0 sec (0 clks)   0%
Max value of stack pointer= 0×000000, avg= 0×000000
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0x000a  10 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a4 cf 00 01       ldw   0×0001,X <——————————save the value X in 0×0001 position
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0x000a  10 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a7 20 f7          jra   0x80a0 <——————– jump…while
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0x000a  10 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a0 ce 00 01       ldw   X,0×0001 <————- restore X with 10 from the ram position 0×0001, to increment it in next instructions
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0x000a  10 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a3 5c             incw  X <—————-increment…x=11
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0x000b  11 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a4 cf 00 01       ldw   0×0001,X <———————–save it(x=11), in 0×0001
0> next    
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0x000b  11 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a7 20 f7          jra   0x80a0 <——————-while
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0x000b  11 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a0 ce 00 01       ldw   X,0×0001 <———-AND SO ON
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0x000b  11 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a3 5c             incw  X
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0x000c  12 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a4 cf 00 01       ldw   0×0001,X
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0x000c  12 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a7 20 f7          jra   0x80a0
0> state
CPU state= OK PC= 0x0080a7 XTAL= 8e+06
Total time since last reset= 3.125e-06 sec (25 clks)
Time in isr = 0 sec (0 clks)   0%
Time in idle= 0 sec (0 clks)   0%
Max value of stack pointer= 0×000000, avg= 0×000000
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0x000c  12 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a0 ce 00 01       ldw   X,0×0001
0> state
CPU state= OK PC= 0x0080a0 XTAL= 8e+06
Total time since last reset= 3.25e-06 sec (26 clks)
Time in isr = 0 sec (0 clks)   0%
Time in idle= 0 sec (0 clks)   0%
Max value of stack pointer= 0×000000, avg= 0×000000
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0x000c  12 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a3 5c             incw  X
0> next
V-IHINZC  Flags= 0×00   0 .  A= 0×00   0 .
0-000000  X= 0x000d  13 .    Y= 0×0000   0 .
SP= 0x07ff [SP+1]= b7 183 .
 ? 0x80a4 cf 00 01       ldw   0×0001,X

Well, I don’t know how the C code is compiled…and I don’t know if the assembler is messing with something…or even if the ucsym have bugs with this, but seems that some work will have to be done…to speed this…

—————————————————————————————–

For now there are 16 known open bugs in SDCC, here you can track the BUGS!

—————————————————————————————–

We will investigate more in the next articles…for example using the compiled file…main.asm and comparing with this…

In mean time if anybody wants to share there thoughts about this feel free to share ;)

Stay tuned ;)

How to use the IP command

The IP command is  a very powerful tool to show/manipulate routing, devices, policy routing and tunnels.

Using the manpages we can have a idea of what it can do…

#man ip

SYNOPSIS
ip [ OPTIONS ] OBJECT { COMMAND | help }
ip [ -force ] -batch filename

OPTIONS := { -V[ersion] | -s[tatistics] | -r[esolve] | -f[amily] { inet | inet6 | ipx | dnet | link } | -o[neline] }
OBJECT := { link | addr | addrlabel | route | rule | neigh | ntable | tunnel | tuntap | maddr | mroute | mrule | monitor | xfrm | netns | l2tp | tcp_metrics }

OPTIONS
-V, -Version
Print the version of the ip utility and exit.
-b, -batch <FILENAME>
Read commands from provided file or standard input and invoke them.  First failure will
cause termination of ip.
-force Don’t terminate ip on errors in batch mode.  If there were any errors during execution of
the commands, the application return code will be non zero.
-s, -stats, -statistics
Output more information.  If the option appears twice or more, the amount of information
increases.  As a rule, the information is statistics or some time values.
-l, -loops <COUNT>
Specify maximum number of loops the ‘ip addr flush’ logic will attempt before giving up.
The default is 10.  Zero (0) means loop until all addresses are removed.
-f, -family <FAMILY>
Specifies the protocol family to use. The protocol family identifier can be one of inet,
inet6, bridge, ipx, dnet or link.  If this option is not present, the protocol family is
guessed from other arguments.  If the rest of the command line does not give enough
information to guess the family, ip falls back to the default one, usually inet or any.
link is a special family identifier meaning that no networking protocol is involved.
-4     shortcut for -family inet.
-6     shortcut for -family inet6.
-B     shortcut for -family bridge.
-D     shortcut for -family decnet.
-I     shortcut for -family ipx.
-0     shortcut for -family link.
-o, -oneline    output each record on a single line, replacing line feeds with the ‘\’ character. This is
convenient when you want to count records with wc(1) or to grep(1) the output.
-r, -resolve
use the system’s name resolver to print DNS names instead of host addresses.

IP – COMMAND SYNTAX
OBJECT
address – protocol (IP or IPv6) address on a device.
addrlabel – label configuration for protocol address selection.
l2tp – tunnel ethernet over IP (L2TPv3).
link – network device.
maddress – multicast address.
monitor – watch for netlink messages.
mroute – multicast routing cache entry.
mrule  – rule in multicast routing policy database.
neighbour – manage ARP or NDISC cache entries.
netns  – manage network namespaces.
ntable – manage the neighbor cache’s operation.
route  – routing table entry.
rule   – rule in routing policy database.
tcp_metrics/tcpmetrics  – manage TCP Metrics
tunnel – tunnel over IP.
tuntap – manage TUN/TAP devices.
xfrm   – manage IPSec policies.

The names of all objects may be written in full or abbreviated form, for exampe address can be abbreviated as addr or just a.

COMMAND
Specifies the action to perform on the object.  The set of possible actions depends on the
object type.  As a rule, it is possible to add, delete and show (or list ) objects, but some
objects do not allow all of these operations or have some additional commands.  The help command
is available for all objects.  It prints out a list of available commands and argument syntax
conventions.

add, delete, list.

If no command is given, some default command is assumed.  Usually it is list or, if the objects
of this class cannot be listed, help.

Some examples:

Show statistics about the interface wlan0.

#ip -s link show wlan0

3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DORMANT qlen 1000
link/ether 78:e4:00:f8:38:6a brd ff:ff:ff:ff:ff:ff
RX: bytes  packets  errors  dropped overrun mcast
838556370  777055   0       0       0       0
TX: bytes  packets  errors  dropped carrier collsns
211549835  611879   0       0       0       0

Show IP adress of the interface wlan0.

#ip addr show wlan0

3: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 78:e4:00:f8:38:6a brd ff:ff:ff:ff:ff:ff
inet 192.168.10.41/24 brd 192.168.10.255 scope global wlan0
inet6 fe80::7ae4:ff:fef8:386a/64 scope link
valid_lft forever preferred_lft forever

Show the routing table:

#ip route show

default via 192.168.10.130 dev wlan0  proto static
192.168.10.0/24 dev wlan0  proto kernel  scope link  src 192.168.10.41

there are better ways to see the routing table with netstat -rn or with route command.

Add an IP(or more) to an interface.

#ip addr add 192.168.1.10/24 brd + dev eth0

#ip addr add 192.168.2.10/24 brd + dev eth0

YES ITS NOT WRONG ;)…eth0 have two ip’s assigned(use spaces on each side of the + sign).

Delete an IP from an interface.

# ip addr del 192.168.3.10/24 dev eth0

Bring an interface up or down

#ip link set eth0 up

#ip link set eth0 down

This article will be updated in future, this command is to much powerful…

Enjoy ;)

How to use dd Comand

This article will explain the options more used in the dd command…

regards

How to create a Mysql/MariaDB Slave Server(2)

This guide require that you have already read the  How to create a Mysql/MariaDB Slave Server(1).

SLAVE

 1. In the config file of Mysql (/etc/mysql/my.cnf)

Edit the configuration file of Mysql  /etc/mysql/my.cnf , with vim for example(you can use what editor you want, nano for example).

vim /etc/mysql/my.cnf
a) Change Bind in [mysqld] section

The slave are listening in 127.0.0.1 , have to be changed to 192.168.1.11, to accept connections from another local network machines.

In [mysqld] ->Basic Settings Section
change bind-address = 127.0.0.1 to bind-address = 192.168.10.11

If the parameter skip-network exists and is enable, replace him by #skip-networking

b) Set Server ID in [mysqld] section

Now that our Database will listen in the 192.168.1.11 and network is enabled, we can configure replication.

In replication mode the servers have a server ID. In section [myslqd].

We will set the server-id of the Slave server to 2 (for a consistency reason).

so in the line #server-id = 1, replace it by server-id = 2

c) Set Binary log file in [mysqld] section

The Slave will get the binlog files  from Master and put them in a temporary log file, relay log.When the Slave execute those query’s from the relay log, it will log the query’s to the Slave binlog.

This file is where goes the modification query’s that are executed in Slave(inserts, updates…), so we call it Slave binlog.

After that…it is a copy of the master!!The slave are constantly fetching the Master binlog to be “equal”.

We have to insert the path and the name of the file that will keep this info.The parameter is log_bin, we can put him like this:

log_bin = /var/log/mysql/mysql-bin.log (we chose the name we want, if it exists we can leave it untouchable). In Maria db there are a different default name for this file: /var/log/mysql/mariadb-bin.

There are another parameter log_bin_index, and for consistency.

log_bin_index = /var/log/mysql/mysql-bin.index

Or in MariaDB

log_bin_index = /var/log/mysql/mariadb-bin.index

Warning: Make sure that the place where the logs will have enough space.

 d) Set  replicate-do-db in [mysqld] section (OPTIONAL)

 replicate-do-db parameter, allows you to define what databases will be replicated.

Use:

 replicate-do-db = <master_database_name>

You can add more lines, each one, with one database.

If omitted, every database (that are in the Master binlog) will be replicated.

If used(replicate-do-db), by default, the binlog-format will not be the same for all query’s.For example, the CREATE TABLE and ALTER TABLE will be of type STATEMENT,and only those databases specified by USE DATABASE term,will be replicated.

Warning: Remember that, if you “lose” log info about a needed database(because are not in a replicate-do-db statement), the slave will not be able to reproduce the exact state of the master, because the Slave will filter information about modifications in that database.

So if you are not a GURU in MySql Databases DON’T USE replicate-do-db!!

We will not use it, but you can learn more about and use it… 

See more about replicate-do-db here.

See more about binlog-format here.

e) Set replicate-ignore-db in [mysqld] section (OPTIONAL)

replicate-ignore-db parameter allows you to define what databases will NOT be replicated to the Slave Server.

replicate-ignore-db = <master_database_name>

If you don’t want to replicate more databases, add more lines with the respectively database you want.

If omitted, every database will be logged(of course if you don’t have replicate-do-db statements).

If used(replicate-ignore-db), by default, the binlog-format will not be the same for all query’s. For example, the CREATE TABLE and ALTER TABLE will be of type STATEMENT, and only those databases specified by the  USE DATABASE term, would not be replicated.

Warning: Remember that, if you “lose” log info about a needed database(because are in a replicate-ignore-db statement or are not in a replicate-do-db statement), the slave will not be able to reproduce the exact state of the master,  because the Slave will filter information about modifications in that database.

So if you are not a GURU in MySql Databases DON’T USE replicate-ignore-db!!

We will not use it, but you can learn more about and use it… 

See more about replicate-ignore-db here .

See more about binlog-format here.

 

f) Set binlog-do-db in [mysqld] section (OPTIONAL)

binlog-do-db parameter, allows you to define what databases will be logged to binlog.

Use:

binlog-do-db = <master_database_name> we are using it in master, in slave its the same name

You can add more lines, each one, with one database.

If omitted, every database will be logged(if you don’t have any restrictions, see 1d) and 1e) ).

If used(binlog-do-db), by default, the binlog-format will not be the same for all query’s.For example, the CREATE TABLE and ALTER TABLE will be of type STATEMENT,and only those databases specified by the term USE DATABASE will be logged.

Warning: Remember that, if you “lose” log info about a needed database(because are not in a binlog-do-db statement or are in binlog-ignore-db), the Slave will not be able to perform a repair in any table of the database, because it don’t have logg info about.

So if you are not a GURU in MySql Databases DON’T USE binlog-do-db!!

We will use it, only with the database we want!!

See more about binlog-do-db  here.

See more about binlog-format here.

g) Set binlog-ignore-db in [mysqld] section (OPTIONAL)

binlog-ignore-db  parameter allows you to define what databases will NOT be logged to binlog.

binlog-ignore-db = <master_database_name> 

If you don’t want that more databases goes to bin_log add more lines with the respectively database you want.

If omitted, every database will be logged(of course if you don’t have binlog-do-db statements).

If used(binlog-ignore-db), by default, the binlog-format will not be the same for all query’s. For example, the CREATE TABLE and ALTER TABLE will be of type STATEMENT, and only those databases specified would not be logged.

Warning: Remember that, if you “lose” log info about a needed database(because are not in a binlog-do-db  statement or are in a binlog-ignore-db statement), the the Slave will not be able to perform a repair in any table of the database, because it don’t have logg info about.

So if you are not a GURU in MySql Databases DON’T USE binlog-ignore-db!!

We will not use it, but you can learn more about and use it…

See more about binlog-ignore-db here .

See more about binlog-format here.

 

Point 1 Done!!

Save the file and exit, in vim do  <esc> : x <enter>

 

Its almost done!!Now we have only to get the info in the Master databases we want to replicate, and perform a CHANGE MASTER TO in the Slave.

MASTER

Get the Dump of all Databases you want

a). Disable Write mode in Master

perform the command in the shell:

mysql -p after that,

mysql> FLUSH TABLES WITH READ LOCK; (the server will stay in read only mode, so it can’t modify the binlog-position).You can stop the Master of course, but in this case the clients will not be able to connect and read info

afther that do ,

mysql> SHOW MASTER STATUS;

Collect the info about Master log file, and Master log position…

Important: Leave this shell open!!If Not the Read lock will be disabled!! 

b) Make a dump of the databases to replicate

Open another shell

perform the command in the shell:

mysqldump -p db_name1  | gzip > db_name1.dump.gz

mysqldump -p db_name2  | gzip > db_name2.dump.gz

When it end you can already enable write in Master…

c) Enabling Write mode in Master

in the shell used in a) do

mysql> UNLOCK TABLES ; The read lock have benn disabled and now you can write already…

or

instead close this shell you have open .

d) Transfer the Dump to the Slave

you can use what tool you want..

echo “put db_name1.dump.gz  db_name2.dump.gz .” | sftp root@192.168.1.11

enter the root password and wait until the end.

we can use better tools like i have mentioned here .

 SLAVE

Import Databases Dump, and set Replication Parameters

a) Create the databases you want to replicate in the Slave Server

cd /root

echo “create database db_name1;” | mysql -p

echo “create database db_name2;” | mysql -p repeat this step until the last one.

b) Import Dump of all Databases to replicate

gunzip  db_name1.dump.gz -c | mysql -p db_name1

gunzip  db_name2.dump.gz -c | mysql -p db_name2

wait to the end of import…

c) Create the Slave Replication Parameters in database

CHANGE MASTER TO changes the parameters that the slave server uses for connecting to the master server, for reading the master binary log, and reading the slave relay log. It also updates the contents of the master.info and relay-log.info files. To use CHANGE MASTER TO, the slave replication threads must be stopped (use STOP SLAVE if necessary).

mysql -p

mysql > stop slave ;

mysql > change master to master_host=’192.168.1.10′, master_user=’Slave_User’, master_password=’Slave_User_Password’, master_log_file=’mysql-bin.000005′, master_log_pos=108;

master_log_file and master_log_pos are the info that we collected from Master(ref :MASTER a) ).

4. Start Slave and see if everything is ok

mysql > start slave ;

mysql > show slave status\G

 you should be able to see :



 Slave_IO_Running: Yes (This thread in responsible for connect to Master and catch his binlog info and put him in the relay log of the Slave)
 Slave_SQL_Running: Yes (This thread read the relay log file and execute the query’s, then log modifications to Slave binlog or not depending on rules that are set)

Last_error: 0

Is all OK, UP and Running!!

Finish!! 

Enjoy ;)

 

< Page 1

GNU Tar : Very useful to create archives and other tasks

The tar tool is very useful when we want to create a Archive with all the structure of a folder inside it, or joint together several files in one archive.

Tar for default preserves the owner,permissions of the files and folders, symlinks, and hardlinks.

If you have a file with the name B, and a hardlink to it with name A, when tar extract the file will create a file with name A, and a hardlink to it with name B(it will exchange by alphabetic order).

Popular options and what they mean :

-c = create tar package

-f = read to/from the named file (instead of the device /tape)

- = redirects the info to stdout or stdin (useful if you want to use a pipe , you have to use -f to specify a file and after -  to specify that the file is stdout)

-d = diff , find differences between archive and folders or files

-g =snapshot file(incremental backup)

-t = list contents of .tar file

-r = append content to a .tar file

-v = verbose (prints the activity to stdout)

-x = extract the contents of .tar file

-z = compress files with gzip format(not so useful for minc files)

-h = besides putting everything else in the tar package, also puts the content pointed by the symlinks(follow symlinks), not the symlinks themselves. Dereference the symlinks.

-l = Check the number of hardlinks dumped for each processed file. If this number does not match the total number of hard links for the file, print a warning message.

 

I use it mostly to copy folders between servers.

1. Creating a tar Archive of alienarena-7.60 .

tar  -cf  alienarena-7.60.tar  alienarena-7.60 – we pass him the name of the tar package(alienarena-7.60.tar) and the folder to copy data from (alienarena-7.60)

archive  alienarena-7.60.tar will be produced.

we can use compression with gzip or other

tar  -czf  alienarena-7.60.tar.gz  alienarena-7.60

 

2. Extrating  a tar Archive of alienarena-7.60 .

tar  -xf  alienarena-7.60.tar we pass him the name of the tar package(alienarena-7.60.tar) and him will extract files do current directory.

archive  alienarena-7.60.tar will be processed and created all the stuff inside.

If we want to extract files to another place we can do

mv alienarena-7.60.tar   /another_place  &&  cd   /another_place &&  tar  -xf  alienarena-7.60.tar or using subshels processes :

tar  -cf  –  alienarena-7.60/   |   ( cd  /Destination_folder/ &&  tar  -xpf  – ); remenber the ” – ” tells tar that the “file” is stdout or stdin and the “(” after the pipe will create a subshel process that will receive trought the pipe the info that have been send by the tar in the parent process shell.

If the file have compression we have to decompress it, if gzip is used, so instead of -xpf use -xzpf.

 

3. Creating an Incremental Backup, NOT Diferential.

Do the incremental backup with -g option, or with  - – listed-incremental=snapshot-file_name

1st backup

Tar to create a incremental backup creates,for the first time that the backup is done, a metadata file called snapshot file which have adicional information about the data, and creates the backup itself.

Is is a total backup, because in the first time the snapshot file does not exist(the backup it self is called a level 0 Backup).

tar -cvvf /home/myacount/Firstbackup.tar -g /home/myacount/SnapShot_file.inc /home/myacount/Directory_Source

2nd Backup is incremental

The second time you run tar it will do a incremental backup.It will read the snapshot file, and will analyze the Directory Source to find new directories or files(the backup it self is called a level 1 Backup).

Because we already have a backup file called  Firstbackup.tar we will create another.

tar -cvvf /home/myacount/Inc_backup.tar -g /home/myacount/SnapShot_file.inc /home/myacount/Directory_Source

you can continue to create incremental backups, remember that you have to create a file with a different name

To create incremental archives automatically you can do a script with the follow ;

#!/bin/bash 

NOW=$(date +”%d-%m-%Y”)

INC=$(date +”%Hh%Mm%Ss”)

FILE=”backup.$NOW-$INC.tar.gz”

tar -czvf  /Some_backup_folder/$FILE  /Folder_to_be_archived  -g  /Some_folder/snapshot_file

You can put this script in the cron tab

 

Incremental dumps depend crucially on time stamps, so the results are unreliable if you modify a file’s time stamps during dumping (e.g., with the ‘–atime-preserve=replace’ option), or if you set the clock backwards.

 

4.  Listing the Content of a Incremental tar Backup

We can list content from level 0 or 1 backup’s , but in level zero backup, the content insider the tarball file is all new, because is the first backup…

We will make a list of the content in the incremental backup(level 1 backups) and in the first backup(level 0).

Level 1 Backup: 

tar -t -v -v -f  an_incremental_backup.tar  -g  /dev/null (since the content is already in the tarball file…we don’t need the snapshot file…we use for example /dev/null)

At the beginning of any line will appear:

Y- if the file is present in the archive

N-if the file is not in this archive

D-if the file is a Directory and is included in the archive

Level 0 Backup:

 tar -t -v -v -f  1st_backup_archive.tar  -g  /dev/null  , but it will not give us the info about Y,N,D above because all info is in the archive(its the first, so what is listed is what it have inside, is like if it mark all with Y,D).

or … because its the first backup :

 tar -t -v -v -f  1st_backup_archive.tar we can treat the first backup(level 0 backup) like a normal tar backup without any additional info.

 

5. Restore an Incremental Backup, NOT Diferential.

Move the tarball files to the place you want and start by numeric order to extract the content :

tar -xf  1st_backup_tarball.tar  -g  /dev/null (you have to start from the beginning- Level 0)

Next:

tar -xf  1st_incremental_backup.tar  -g  /dev/null (first level 1)

Next:

 tar -xf  2nd_incremental_backup.tar  -g  /dev/nul (second level 1)

And so on until the last…

References:

http://www.gnu.org/software/tar/manual/html_node/Incremental-Dumps.html

 

Enjoy ;)

 

Netcat: The TCP/IP Swiss Army Knife

The netcat tool is a very powerful tool. It is some times called the Swiss Army Knife because of it characteristics. But…it have a security problem, the data will not be encrypted because it have no encryption mechanisms, so use it wisely.

You can find all the options for nc typing man nc in your console.

I use it to send data between servers.

The examples suppose that you are root!!!

1. Dumping a MySql/MariaDB Database from a Server A to a Server B. 

Imagine that you have to send data from Server A to Server B, then:

1- In server A prepare the commands, and execute them only after executing the commands in Server B ( B will be a nc Server).

mysqldump -p mydatabase | nc -q 0 192.168.1.12 3456(-q 0 will close the socket after 0 seconds of ending the job, 192.168.1.12 3456, is the IP of the server B, and the port that we will choose in serve B.

If the database is a large one we can use compression, and instead of the above command we use:

mysqldump -p mydatabase | gzip -c | nc -q 0 192.168.1.12 3456 (gzip -c, will write output to stdout)

Go to the server B…

2- In server B we have to put netcat listen in one port, in this case…

nc -l -p 3456 | mysql -p mydatabase(-l to listen(is a server…it have to listen in some port), -p to indicate the port we will use).The nc server will listen in port 3456 in all the network interfaces.

If we are using compression, instead of the above command we use:

nc -l -p 3456 | gunzip -c | mysql -p mydatabase

After this commands, in Server A and server B:

1st execute the command in Server B, it will ask you for the password from the root user to access database,…insert it.

2nd execute the command in server A, it will ask you for the password from the root user to access database,…insert it.

Remember that I have assumed that the database mydatabase in server B already exists…if not, you have to create it, with mysql-client, do create database mydatabase; after this do all the procedures above.

2. Cloning two partitions that are unmounted from Server A to Server B.

Now that we know how to use netcat, we can improve our method, by switching the points 1,2, so :

we will start with the point 2. In server B we  will do ifconfig(to know the IP of this server), remember that you don’t have encryption, using public networks can be dangerous, depending on how sensible is your data …we will use a private network.

Imagine that the IP of Server B is 192.168.1.12.

Since this will put on network a great amount of information we will use compression with gzip.

In Server B:

nc -l -p 3456 | gunzip -c | dd of=/dev/sda2(sda2 is an example of a partition, you can use what you have and want)

Don’t execute now the command, wait for the other part, in server A. If not the server starts listening and wasting resources doing nothing…

Now in Server A:

dd if=/dev/sda1 | gzip -c | nc -q 0 192.168.1.12 3456

1st execute the command in Server B (it will be a nc server listen on 3456 port).

2nd execute the command in server A.

This procedure can take a long time depending on how huge is sda1 in Sever A.The image will be of the size of the partition…

Decreasing the size of the Image, if we want to store it or to reduce the network bandwidth for example:

Remember, the dd command will make an exact binary copy of all content in the partition…occupied content + free content…all content(example:imagine that you have a partition with 100GB of space…the dd command will make an image of 100GB, without know about the free space…it doesn’t mater to him).

This will put, in the most part of the cases, a tremendous amount of work for nothing(making a copy of free space…), and we can NOT do something about.

But if we want to store the image in some place, is not a good idea to have a image with , 50% free space, for example, and the rest occupied…so the idea is a deep compression of  the free space.

Imagine that you have 50% of the partition free(you have seen it with df command), we will create a normal file with, 50% of Space(the entire free space, or closest to it, how much close it is, better result we have), but this file will have only zero’s(0).

This will allow the compression mechanism do make a deeply good compression of this file(it only have zeros, all the characters are equal)…and so the entire amount of the image, the occupied space + freespace(file with zeros) , will have a lower size.

So first create the zero’s file and after that make a image with compression(in a partition with 100GB and 50% occupied)…

In Server A do :

dd bs=1M if=/dev/zero of=/dev/sda1/home/zeros_file count=50000(50GB, because blocksize is 1M)

After that you can store the image where you want with :

dd if=/dev/sda1 | gzip -c | dd of=/dev/sdax/image_file.gz(where x is the partition that will store this image)

or after the creation of the zeros_file use the netcat like have been done above.

After that remove the zeros_file in the destination  partitionThe destination partition will have the same size as the source one.

Remember using dd is dangerous, use it with precaution. The author of this article will not be responsible for the damages…

See more about dd command here.

3. Forward requests from port A to Port B, in this case in the same Server but we can use different Servers.

Example: From port 8080 to 80

Imagine that you have a HTTP server on port 80, and want to use the 8080 port too.

A request that will be made to 8080 will be forward to port 80, then the request come to 8080 again and will be sent to the person that made the request.

1-We will have a nc server that listens in 8080 port, it will send the requests through a pipe to a nc HTTP client(port 80).

2-The netcat receive the request sent trough the pipe and make a connection to the HTTP server in port 80.

3-The HTTP server reply trough the netcat, and the netcat send the response to the fifo.

4-Then , the server that is listen in 8080 receives the response trough the fifo and send it to destination.

mkfifo /tmp/fifo (creating the fifo in /tmp, you can call it wherever you want).

nc -lv -v -p 8080 < /tmp/fifo  |  nc  -v -v  localhost 80 > /tmp/fifo , this line will do the trick.

But, wait….this only make a request…we have to make a while loop!!

In a Script:

#!/bin/bash

mkfifo /tmp/fifo

for (( ; ; )) ; do

 nc -lv -v -p 8080 < /tmp/fifo  |  nc  -v -v  localhost 80 > /tmp/fifo ;

done

or 

while true ;do

 nc -lv -v -p 8080 < /tmp/fifo  |  nc  -v -v  localhost 80 > /tmp/fifo ;

done 

You can put this script in a RunLevel…

I’m not sure,but I think this will run well if you have few requests, if you have too many requests, this forward port system may get slow to responses.

Perhaps you find a better solution…because you will have to make bind in 8080, a pipe, redirections…for each request.

 

Enjoy ;)

How to create a Mysql/MariaDB Slave Server(1)

We wil create a MySql/MariaDB Slave Server in a private Network(Without Encryption).

Assumptions :

  • We are using Debian Squeeze on both , Master and Slave Servers.
  • One MySql/MariaDB Master Database (Only one server not two in Master/Master).
  • We will use Same Network, and more important…private network, since the info that will come from the Master to Slave have no Encryption (we can do something about, but not in this article).
  • The Master db Server have a private IP 192.168.1.10.
  • The Slave  db Server have a private IP 192.168.1.11.
  • The Slave already have the mysql-server meta-package installed, and of course mysql-client too.
  • We have the root password to connect to Master/Slave db Servers from “localhost”.
  • The received connection will be done on the standard port 3306, for both Master and Slave.
  • Master and Slave only listens in the loopback address(127.0.0.1).
  • You have the root password of the Master ans Slave.

 

With the root user:

Master

1. In the config file of Mysql(/etc/mysql/my.cnf)

Edit the configuration file of Mysql /etc/mysql/my.cnf , with vim for example(you can use what editor you want, nano for example).

vim /etc/mysql/my.cnf

a) Change Bind in [mysqld] section

The master are listening in 127.0.0.1 , have to be changed to 192.168.1.10, to accept connections from another local network machines.

In [mysqld] ->Basic Settings Section
change bind-address = 127.0.0.1 to bind-address = 192.168.10.10

If the parameter skip-network exists and is enable, replace him by #skip-networking

b) Set Server ID in [mysqld] section

Now that our Database will listen in the 192.168.1.10 and network is enabled, we can configure replication.

In replication mode the servers have a server ID(to control replication info to any Slave).In section [myslqd].

We will set the server-id of the master server to 1 (for a consistency reason).

so in the line  #server-id = 1, replace it by server-id = 1

c) Set Binary log file in [mysqld] section

This file is where goes the modification query’s that are made in master(inserts, updates…).The slave will come get those log files on Master, and will execute the query’s that are in this log files. After that…it is a copy of the master!! This is the replication itself.

We have to insert the path and the name of the file that will keep this info.The parameter is  log_bin.

we can put him like this:

log_bin = /var/log/mysql/mysql-bin.log (we chose the name we want, if it exists we can let it untouchable).In Maria db there are a different default name for this file: /var/log/mysql/mariadb-bin.

There are another parameter log_bin_index, and for consistency.

log_bin_index = /var/log/mysql/mysql-bin.index

Or in MariaDB

 log_bin_index = /var/log/mysql/mariadb-bin.index

Warning: Make sure that the path where the logs goes to have suficient space to this log files.

d) innodb_flush_log_at_trx_commit and sync_binlog in [mysqld] section

“For the greatest possible durability and consistency in a replication setup using InnoDB with transactions, use innodb_flush_log_at_trx_commit = 1 and sync_binlog = 1 in your master server my.cnf file.”

You can learn more about here.

e) Set binlog-do-db in [mysqld] section (OPTIONAL)

binlog-do-db parameter, allows you to define what databases will be logged to binlog.

Use:

binlog-do-db = <master_database_name>

You can add more lines, each one, with one database.

If omitted, every database will be logged.

If used(binlog-do-db), by default, the binlog-format will not be the same for all query’s.For example, the CREATE TABLE and ALTER TABLE will be of type STATEMENT,and only those databases specified by the term USE DATABASE will be logged.

Warning: Remember that, if you “lose” log info about a needed database(because are not in a binlog-do-db statement), the slave will not be able to reproduce the exact state of the master, because the binlog lack information about modifications in that database.More…the Master will not be able to perform a repair in any table of the database, because it don’t have logg info about.

So if you are not a GURU in MySql Databases DON’T USE binlog-do-db!!

We will use it, only with the database we want!!

See more about binlog-do-db  here.

See more about binlog-format here.

f) Set binlog-ignore-db in [mysqld] section (OPTIONAL)

binlog-ignore-db  parameter allows you to define what databases will NOT be logged to binlog.

binlog-ignore-db = <master_database_name>

If you don’t want that more databases goes to bin_log add more lines with the respectively database you want.

If omitted, every database will be logged(of course if you don’t have binlog-do-db statements).

If used(binlog-ignore-db), by default, the binlog-format will not be the same for all query’s. For example, the CREATE TABLE and ALTER TABLE will be of type STATEMENT, and only those databases specified by the term USE DATABASE would not be logged.

Warning: Remember that, if you “lose” log info about a needed database(because are not in a binlog-do-db  statement or are in a binlog-ignore-db statement), the slave will not be able to reproduce the exact state of the master, because the binlog lack information about modifications in that database.More…the Master will not be able to perform a repair in any table of the database, because it don’t have logg info about.

So if you are not a GURU in MySql Databases DON’T USE binlog-ignore-db!!

We will not use it, but you can learn more about and use it…

See more about binlog-ignore-db here .

See more about binlog-format here.

 

Point 1 Done!!

Save the file and exit, in vim do  <esc> : x <enter>

2. In the Database

 

Enter mysql -p in the shell

a) Change Grants for actual users(Not for MySql/MariaDB root).

We will put the server listening in 192.168.1.10. If we have users with grants, others than mysql root, in the database, they have of course grants to connect via localhost(because our server are listening in localhost until now 127.0.0.1).

Nothing to be Done for now!!

b) Creating a Replication User

This user  will be used by the Slave Server to connect to the master and catch the binlogs required to perform the replication.

mysql> create user ‘Slave_User’@’192.168.1.11’  identified  by  ‘Slave_User_Password’ ;
mysql> grant  replication  slave  on  * . *  to  ‘Slave_User’@’192.168.1.11’ ;
mysql> flush  privileges ; 

In this case, the Slave User will have grants in all databases only for replication….Note that we are replicating all databases, we are not using filter options above it configuration file /etc/mysql/my.cnf(described above in points 1.e),1.f) ).

or in only two lines(if the user don’t exists will be created automatically):

mysql> grant  replication  slave  on * . *  to ‘Slave_User’@’192.168.1.11’ identified by ‘Slave_Password’;

 mysql>  flush  privileges;

See more about Adding User Accounts here .

See more about Grants here .

The work in Master, FOR NOW, is finished…we will come back to master again after configure the Slave Server to get all the info that the databases have inside.

 

Page 2 >