Untitled... Still...

A > B, B > A, Sites Basics

In case that you have 2 or more of geo locations, infrastructure you may need can look something like this ?


                                                             [___Internet___]                                                        
  _____________________________________________________             |             _____________________________________________________ 
 | [ an-node01 ]                                       |            |            |                                       [ an-node02 ] |
 |                       ____________    ______________|        ____|____        |______________    ____________                       |
 |                      |    vbr2    |--| bond2        |       | [ IFN ] |       |        bond2 |--|   vbr2     |                      |
 |  _________________   | 10.255.0.1 |  | ______       |      _|_________|_      |       ______ |  | 10.255.0.2 |  ................... |
 | | [ vm0001-dev ]  |  |____________|  || eth2 =--\   |     |   Switch 1  |     |   /--= eth2 ||  |____________|  :  [ vm0001-dev ] : |
 | | [ Dev Server ]  |    | | : :       ||_____|    \--=-----|_____________|-----=--/    |_____||       | | : :    :  [ Dev Server ] : |
 | |           ______|    | | : :       | ______    /--=-----|   Switch 2  |-----=--\    ______ |       | | : :    :.......          : |
 | |          | eth0 =----/ | : :       || eth5 =--/   |     |_____________|     |   \--= eth5 ||       | | : :----= eth0 :          : |
 | |          |_____||      | : :       ||_____|       |                         |       |_____||       | | :      ::.....:          : |
 | |      10.254.0.1 |      | : :       |______________|                         |______________|       | | :      :                 : |
 | |_________________|      | : :        ______________|                         |______________        | | :      :.................: |
 |                          | : :       | bond1        |        _________        |        bond1 |       | | :                          |
 |  _________________       | : :       |   10.10.0.1  |       | [ SN  ] |       | 10.10.0.2    |       | | :      ................... |
 | | [ vm0002-web ]  |      | : :       | ______       |      _|_________|_      |       ______ |       | | :      :  [ vm0002-web ] : |
 | | [ Web Server ]  |      | : :       || eth1 =--\   |     |   Switch 1  |     |   /--= eth1 ||       | | :      :  [ Web Server ] : |
 | |           ______|      | : :       ||_____|    \--=-----|_____________|-----=--/    |_____||       | | :      :.......          : |
 | |          | eth0 =------/ : :       | ______    /--=-----|   Switch 2  |-----=--\    ______ |       | | :------= eth0 :          : |
 | |          |_____||        : :       || eth4 =--/   |     |_____________|     |   \--= eth4 ||       | |        ::.....:          : |
 | |      10.254.0.2 |        : :       ||_____|       |                         |       |_____||       | |        :                 : |
 | |_________________|        : :       |______________|                         |______________|       | |        :.................: |
 |                            : :        ______________|                         |______________        | |                            |
 | ...................        : :       | bond0        |        _________        |        bond0 |       | |         _________________  |
 | : [ vm0003-db  ]  :        : :       |   10.20.0.1  |       | [ BCN ] |       | 10.20.0.2    |       | |        |  [ vm0003-db  ] | |
 | : [ DB Server  ]  :        : :       | ______       |      _|_________|_      |       ______ |       | |        |  [ DB Server  ] | |
 | :          .......:        : :       || eth0 =--\   |  /--|   Switch 1  |--\  |   /--= eth0 ||       | |        |______           | |
 | :          : eth0 =--------: :       ||_____|    \--=--+--|_____________|--+--=--/    |_____||       | \--------= eth0 |          | |
 | :          :.....::          :       | ______    /--=--+--|   Switch 2  |--+--=--\    ______ |       |          ||_____|          | |
 | :                 :          :       || eth3 =--/   |  |  |_____________|  |  |   \--= eth3 ||       |          | 10.254.0.3      | |
 | :.................:          :       ||_____|       |  |     |       |     |  |       |_____||       |          |_________________| |
 |                              :       |______________|  |     |       |     |  |______________|       |                              |
 | ...................          :                      |  |     |       |     |  |                      |           _________________  |
 | : [ vm0004-win ]  :          :                      |  |     |       |     |  |                      |          |  [ vm0004-win ] | |
 | : [ MS Server  ]  :          :                      |  |     |       |     |  |                      |          |  [ MS Server  ] | |
 | :          .......:          :                      |  |     |       |     |  |                      |          |______           | |
 | :          : NIC0 =----------:                      |  |     |       |     |  |                      \----------= NIC0 |          | |
 | :          :.....::                           ______|  |     |       |     |  |______                           ||_____|          | |
 | :                 :                  _____   | IPMI =--/     |       |     \--= IPMI |   _____                  | 10.254.0.4      | |
 | :.................:                 [_BMC_]--|_____||        |       |        ||_____|--[_BMC_]                 |_________________| |
 |                                                     |        |       |        |                                                     |
 |                                 ______ ______       |        |       |        |       ______ ______                                 |
 |                                | PSU1 | PSU2 |      |        |       |        |      | PSU2 | PSU1 |                                |
 |________________________________|______|______|______|        |       |        |______|______|______|________________________________|
                                       || ||                ____|_     _|____                || ||                                      
                                       || ||               | PDU1 |   | PDU2 |               || ||                                      
                                       || ||               |______|   |______|               || ||                                      
                                       || ||                 || ||     || ||                 || ||                                      
                                       || \\===[ Power 1 ]===// ||     || \\===[ Power 1 ]===// ||                                      
                                       \\======[ Power 2 ]======||=====//                       ||                                      
                                                                \\=============[ Power 2 ]======//


#### Storage part

 _________________________________________________                 _________________________________________________ 
| [ an-node01 ]                                   |               |                                   [ an-node02 ] |
|  ________       __________                      |               |                      __________       ________  |
| [_disk_1_]--+--[_/dev/sda_]                     |               |                     [_/dev/sda_]--+--[_disk_1_] |
|  ________   |    |   ___________    _______     |               |     _______    ___________   |    |   ________  |
| [_disk_2_]--+    +--[_/dev/sda1_]--[_/boot_]    |               |    [_/boot_]--[_/dev/sda1_]--+    +--[_disk_2_] |
|  ________   |    |   ___________    _______     |               |    ________    ___________   |    |   ________  |
| [_disk_3_]--+    +--[_/dev/sda2_]--[__swap_]    |               |   [__swap__]--[_/dev/sda2_]--+    +--[_disk_3_] |
|  ________   |    |   ___________    ___         |               |         ___    ___________   |    |   ________  |
| [_disk_4_]--/    +--[_/dev/sda3_]--[_/_]        |               |        [_/_]--[_/dev/sda3_]--+    \--[_disk_4_] |
|                  |   ___________                |               |                ___________   |                  |
|                  +--[_/dev/sda5_]------------\  |               |  /------------[_/dev/sda5_]--+                  |
|                  |   ___________             |  |               |  |             ___________   |                  |
|                  +--[_/dev/sda6_]----------\ |  |               |  | /----------[_/dev/sda6_]--+                  |
|                  |   ___________           | |  |               |  | |           ___________   |                  |
|                  \--[_/dev/sda7_]--------\ | |  |               |  | | /--------[_/dev/sda7_]--/                  |
|        _______________    ____________   | | |  |               |  | | |   ____________    _______________        |
|    /--[_Clustered_LVM_]--[_/dev/drbd2_]--/ | |  |               |  | | \--[_/dev/drbd2_]--[_Clustered_LVM_]--\    |
|   _|__                     |   _______     | |  |               |  | |      |   _______                    __|_   |
|  [_PV_]                    \--{_bond1_}    | |  |               |  | |      \--{_bond1_}                  [_PV_]  |
|   _|________                               | |  |               |  | |                               ________|_   |
|  [_an02-vg0_]                              | |  |               |  | |                              [_an02-vg0_]  |
|    |   ________________________    ....... | |  |               |  | |  _____     ________________________   |    |
|    +--[_/dev/an02-vg0/vm0003_1_]---:.vm3.: | |  |               |  | | [_vm3_]---[_/dev/an02-vg0/vm0003_1_]--+    |
|    |   ________________________    ....... | |  |               |  | |  _____     ________________________   |    |
|    \--[_/dev/an02-vg0/vm0004_1_]---:.vm4.: | |  |               |  | | [_vm4_]---[_/dev/an02-vg0/vm0004_1_]--/    |
|          _______________    ____________   | |  |               |  | |   ____________    _______________          |
|      /--[_Clustered_LVM_]--[_/dev/drbd1_]--/ |  |               |  | \--[_/dev/drbd1_]--[_Clustered_LVM_]--\      |
|     _|__                     |   _______     |  |               |  |      |   _______                    __|_     |
|    [_PV_]                    \--{_bond1_}    |  |               |  |      \--{_bond1_}                  [_PV_]    |
|     _|________                               |  |               |  |                               ________|_     |
|    [_an01-vg0_]                              |  |               |  |                              [_an01-vg0_]    |
|      |   ________________________     _____  |  |               |  | .......    ________________________   |      |
|      +--[_/dev/an01-vg0/vm0001_1_]---[_vm1_] |  |               |  | :.vm1.:---[_/dev/an01-vg0/vm0001_1_]--+      |
|      |   ________________________     _____  |  |               |  | .......    ________________________   |      |
|      \--[_/dev/an01-vg0/vm0002_1_]---[_vm2_] |  |               |  | :.vm2.:---[_/dev/an01-vg0/vm0002_1_]--/      |
|            _______________    ____________   |  |               |  |   ____________    _______________            |
|        /--[_Clustered_LVM_]--[_/dev/drbd0_]--/  |               |  \--[_/dev/drbd0_]--[_Clustered_LVM_]--\        |
|       _|__                     |   _______      |               |       |   _______                    __|_       |
|      [_PV_]                    \--{_bond1_}     |               |       \--{_bond1_}                  [_PV_]      |
|       _|__________                              |               |                              __________|_       |
|      [_shared-vg0_]                             |               |                             [_shared-vg0_]      |
|       _|_________________________               |               |               _________________________|_       |
|      [_/dev/shared-vg0/lv_shared_]              |               |              [_/dev/shared-vg0/lv_shared_]      |
|        |   ______    _________                  |               |                  _________    ______   |        |
|        \--[_GFS2_]--[_/shared_]                 |               |                 [_/shared_]--[_GFS2_]--/        |
|                                          _______|   _________   |_______                                          |
|                                         | bond1 =--| Storage |--= bond1 |                                         |
|                                         |______||  | Network |  ||______|                                         |
|_________________________________________________|  |_________|  |_________________________________________________|

Let's look at two scenarios showing how locks are handled using CPG;

The cluster starts up cleanly with two members.
Both members are able to start service:foo.
Both want to start it, but need a lock from DLM to do so.
The an-node01 member has its totem token, and sends its request for the lock.
DLM issues a lock for that service to an-node01.
The an-node02 member requests a lock for the same service.
DLM rejects the lock request.
The an-node01 member successfully starts service:foo and announces this to the CPG members.
The an-node02 sees that service:foo is now running on an-node01 and no longer tries to start the service.

Storage part:

The two members want to write to a common area of the /shared GFS2 partition.
The an-node02 sends a request for a DLM lock against the FS, gets it.
The an-node01 sends a request for the same lock, but DLM sees that a lock is pending and rejects the request.
The an-node02 member finishes altering the file system, announces the changed over CPG and releases the lock.
The an-node01 member updates its view of the filesystem, requests a lock, receives it and proceeds to update the filesystems.
It completes the changes, annouces the changes over CPG and releases the lock.

Messages can only be sent to the members of the CPG while the node has a totem token from corosync.