اجرای کلاستر اوراکل در داکر(Oracle RAC 21c)

برای نصب Oracle RAC به منظور استفاده در محیط آموزشی و یا اجرای بعضی از تستها، معمولا از Oracle Virtual Box و یا VMware workstation استفاده می کنیم.

برای این کار باید به تعداد نودهای کلاستر، ماشین مجازی ایجاد کرده و بر روی هر کدام از این ماشینها، سیستم عاملی را نصب کنیم و در نهایت اقداماتی را در هر کدام از این سیستم عاملها انجام دهیم تا شرایط برای نصب کلاستر فراهم شود. پیکربندی کلاستر در محیط VM نسبتا زمانبر است و شاید کمی پیچیده هم باشد.

 البته در این زمینه، VM تنها گزینه ما نیست و استفاده از Docker می تواند به عنوان انتخابی دیگر، ایرادات ذکر شده را از بین ببرد.

اوراکل از نسخه 12c امکان اجرای Oracle RAC در داکر را برای محیط تست و develop فراهم کرده و در نسخه 21c، از اجرای Oracle RAC در محیط docker آن هم به صورت عملیاتی پشتیبانی می کند.

اجرای Oracle RAC در داکر، می تواند با استفاده از یک یا چند هاست انجام شود که در این متن صرفا از یک هاست استفاده خواهیم کرد و راه اندازی آن در چند هاست را به زمانی دیگر موکول می کنیم.

سیستم عاملی که از آن استفاده کرده ایم، Oracle Linux نسخه 7 می باشد و قرار است Oracle RAC نسخه 21c را در این محیط اجرا کنیم. این کار با کمک سه container انجام خواهد شد که یکی از آنها برای DNS server و دو container دیگر هم هر کدام نودهای کلاستر را تشکیل خواهند داد.

در ابتدا مراحل پیکربندی را مرور می کنیم:

1.نصب و اجرای داکر

2.آماده سازی داکر هاست

3.تنظیم Docker Network

4.نصب git و کلون Oracle Repository

5.ایجاد container برای DNS

6.ایجاد container برای راه اندازی نود اول کلاستر

7.اضافه کردن نود دوم کلاستر

نصب و اجرای داکر

با توجه به آنکه نحوه “نصب داکر در اوراکل لینوکس” قبلا توضیح داده شده از تکرار آن در این متن می پرهیزیم و صرفا با دستور زیر، وضعیت سرویس docker را بررسی می کنیم:

[root@OEL7 ~]# systemctl status docker
● docker.service - Docker Application Container Engine Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled) Active: active (ruing) since Wed 2022-10-05 13:43:33 +0330; 8ms ago Docs: https://docs.docker.com Main PID: 44314 (dockerd) Tasks: 30 Memory: 84.1M CGroup: /system.slice/docker.service └─44314 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --cp...

آماده سازی داکر هاست

برای آماده سازی docker host، مراحل زیر را انجام خواهیم داد.

1.پارامترهای زیر را در فایل sysctl.conf تنظیم می کنیم:

[root@OEL7 ~]# vi /etc/sysctl.conf
fs.aio-max-nr=1048576
fs.file-max = 6815744
net.core.rmem_max = 4194304
net.core.rmem_default = 262144
net.core.wmem_max = 1048576
net.core.wmem_default = 262144
net.core.rmem_default = 262144

و با اجرای دستور sysctl -p این تغییرات را اعمال می کنیم.

2.پارامتر cpu-rt-runtime=950000– را به صورت زیر به فایل docker.service اضافه می کنیم:

[root@OEL7 ~]# vi /usr/lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --cpu-rt-runtime=950000

بعد از اعمال این تغییر، باید سرویس داکر را restart کنیم:

[root@OEL7 ~]# systemctl daemon-reload
[root@OEL7 ~]# systemctl stop docker
[root@OEL7 ~]# systemctl restart docker

3.فایروال را stop و غیرفعال می کنیم:

[root@OEL7 ~]# systemctl stop firewalld.service
[root@OEL7 ~]# systemctl disable firewalld.service

ایجاد Docker Network

همانطور که می دانید، برای راه اندازی کلاستر حداقل به دو کارت شبکه public و private نیاز خواهیم داشت، در این مرحله، این دو کارت شبکه را ایجاد می کنیم:

[root@OEL7 ~]# docker network create --driver=bridge --subnet=10.0.20.0/24 rac_pub_nw1
710ae5777ba3cb207b6b0b49ec37e89a0554347bae054d5017af5fa3a35a03cb
[root@OEL7 ~]# docker network create --driver=bridge --subnet=192.168.10.0/24 rac_priv_nw1
f3824262f932161851fde89da802c29a9323f4ff76c1bb8c88fe42b6546c7880

نصب git و کلون Oracle Repository

در قدم بعدی، git را نصب می کنیم:

[root@OEL7 ~]# yum install -y git

بعد از نصب gitء، Oracle repository را در مسیر OracleRepoGIT/ کلون می کنیم:

[root@OEL7 ~]# mkdir /OracleRepoGIT
[root@OEL7 ~]# cd /OracleRepoGIT
[root@OEL7 OracleRepoGIT]# git clone https://github.com/oracle/docker-images.git
Cloning into 'docker-images'...
remote: Enumerating objects: 16096, done.
remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 16096 (delta 3), reused 1 (delta 0), pack-reused 16081
Receiving objects: 100% (16096/16096), 10.47 MiB | 3.86 MiB/s, done.
Resolving deltas: 100% (9512/9512), done.

با اجرای این دستور، فایلها و دایرکتوری های زیر ایجاد خواهند شد:

[root@OEL7 OracleRepoGIT]# ls -l
drwxr-xr-x 32 root root 4096 Oct 5 13:48 docker-images
[root@OEL7 docker-images]# ll
drwxr-xr-x 6 root root 110 Oct 5 13:48 Archive
-rw-r--r-- 1 root root 3359 Oct 5 13:48 CODE_OF_CONDUCT.md
-rw-r--r-- 1 root root 1338 Oct 5 13:48 CODEOWNERS
drwxr-xr-x 3 root root 53 Oct 5 13:48 Contrib
-rw-r--r-- 1 root root 9695 Oct 5 13:48 CONTRIBUTING.md
drwxr-xr-x 3 root root 16 Oct 5 13:48 GraalVM
-rw-r--r-- 1 root root 1844 Oct 5 13:48 LICENSE.txt
drwxr-xr-x 3 root root 33 Oct 5 13:48 NoSQL
drwxr-xr-x 4 root root 76 Oct 5 13:48 OracleAccessManagement
drwxr-xr-x 3 root root 59 Oct 5 13:48 OracleBI
drwxr-xr-x 4 root root 59 Oct 5 13:48 OracleCloudInfrastructure
drwxr-xr-x 2 root root 23 Oct 5 13:48 OracleCoherence
drwxr-xr-x 4 root root 56 Oct 5 13:48 OracleDatabase
drwxr-xr-x 4 root root 57 Oct 5 13:48 OracleEssbase
drwxr-xr-x 4 root root 57 Oct 5 13:48 OracleFMWInfrastructure
drwxr-xr-x 4 root root 49 Oct 5 13:48 OracleGoldenGate
drwxr-xr-x 4 root root 74 Oct 5 13:48 OracleHTTPServer
drwxr-xr-x 5 root root 91 Oct 5 13:48 OracleIdentityGoveance
drwxr-xr-x 4 root root 63 Oct 5 13:48 OracleInstantClient
drwxr-xr-x 6 root root 62 Oct 5 13:48 OracleJava
drwxr-xr-x 4 root root 63 Oct 5 13:48 OracleLinuxDevelopers
drwxr-xr-x 3 root root 42 Oct 5 13:48 OracleManagementAgent
drwxr-xr-x 3 root root 33 Oct 5 13:48 OracleOpenJDK
drwxr-xr-x 3 root root 77 Oct 5 13:48 OracleRestDataServices
drwxr-xr-x 4 root root 91 Oct 5 13:48 OracleSOASuite
drwxr-xr-x 5 root root 109 Oct 5 13:48 OracleUnifiedDirectory
drwxr-xr-x 4 root root 94 Oct 5 13:48 OracleUnifiedDirectorySM
drwxr-xr-x 3 root root 22 Oct 5 13:48 OracleVeridata
drwxr-xr-x 3 root root 42 Oct 5 13:48 OracleWebCenterContent
drwxr-xr-x 3 root root 42 Oct 5 13:48 OracleWebCenterPortal
drwxr-xr-x 3 root root 59 Oct 5 13:48 OracleWebCenterSites
drwxr-xr-x 4 root root 74 Oct 5 13:48 OracleWebLogic
-rw-r--r-- 1 root root 3127 Oct 5 13:48 README.md

ایجاد container برای DNS

همانطور که می دانید، برای داشتن scan ipها، بهتر است از DNS سرور استفاده کنیم، با انجام روال زیر، ابتدا image و سپس containerای را برای DNS ایجاد خواهیم کرد.

در ابتدا دو فایل zonefile و reversezonefile موجود در مسیر OracleRepoGIT را به شکل زیر اصلاح می کنیم:

[root@OEL7 OracleRepoGIT]# cd /OracleRepoGIT/docker-images/OracleDatabase/RAC/OracleDNSServer/dockerfiles/latest
[root@OEL7 latest]# vi zonefile

$TTL 86400
@ IN SOA ###DOMAIN_NAME###. root ( 2014090401 ; serial 3600 ; refresh 1800 ; retry 604800 ; expire 86400 ) ; minimum
; Name server's IN NS ###DOMAIN_NAME###.
; Name server hostname to IP resolve. IN A ###RAC_DNS_SERVER_IP###
; Hosts in this Domain
###HOSTNAME### IN A ###RAC_DNS_SERVER_IP###
###RAC_NODE_NAME_PREFIX###1 IN A ###RAC_PUBLIC_SUBNET###.151
###RAC_NODE_NAME_PREFIX###2 IN A ###RAC_PUBLIC_SUBNET###.152
###RAC_NODE_NAME_PREFIX###3 IN A ###RAC_PUBLIC_SUBNET###.153
###RAC_NODE_NAME_PREFIX###4 IN A ###RAC_PUBLIC_SUBNET###.154
###RAC_NODE_NAME_PREFIX###1-vip IN A ###RAC_PUBLIC_SUBNET###.161
###RAC_NODE_NAME_PREFIX###2-vip IN A ###RAC_PUBLIC_SUBNET###.162
###RAC_NODE_NAME_PREFIX###3-vip IN A ###RAC_PUBLIC_SUBNET###.163
###RAC_NODE_NAME_PREFIX###4-vip IN A ###RAC_PUBLIC_SUBNET###.164
###RAC_NODE_NAME_PREFIX###-scan IN A ###RAC_PUBLIC_SUBNET###.171
###RAC_NODE_NAME_PREFIX###-scan IN A ###RAC_PUBLIC_SUBNET###.172
###RAC_NODE_NAME_PREFIX###-scan IN A ###RAC_PUBLIC_SUBNET###.173
###RAC_NODE_NAME_PREFIX###-gns1 IN A ###RAC_PUBLIC_SUBNET###.175
###RAC_NODE_NAME_PREFIX###-gns2 IN A ###RAC_PUBLIC_SUBNET###.176

[root@OEL7 latest]# mv reversezonefile reversezonefile-old
[root@OEL7 latest]# vi reversezonefile

$TTL 86400
@ IN SOA ###DOMAIN_NAME###. root.###DOMAIN_NAME###. ( 2014090402 ; serial 3600 ; refresh 1800 ; retry 604800 ; expire 86400 ) ; minimum
; Name server's ###HOSTNAME_IP_LAST_DIGITS### IN NS ###DOMAIN_NAME###.
; Name server hostname to IP resolve. IN PTR ###HOSTNAME###.###DOMAIN_NAME###.
; Second RAC Cluster on Same Subnet on Docker
151 IN PTR ###RAC_NODE_NAME_PREFIX###1.###DOMAIN_NAME###.
152 IN PTR ###RAC_NODE_NAME_PREFIX###2.###DOMAIN_NAME###.
153 IN PTR ###RAC_NODE_NAME_PREFIX###3.###DOMAIN_NAME###.
154 IN PTR ###RAC_NODE_NAME_PREFIX###4.###DOMAIN_NAME###.
161 IN PTR ###RAC_NODE_NAME_PREFIX###1-vip.###DOMAIN_NAME###.
162 IN PTR ###RAC_NODE_NAME_PREFIX###2-vip.###DOMAIN_NAME###.
163 IN PTR ###RAC_NODE_NAME_PREFIX###3-vip.###DOMAIN_NAME###.
164 IN PTR ###RAC_NODE_NAME_PREFIX###4-vip.###DOMAIN_NAME###.
171 IN PTR ###RAC_NODE_NAME_PREFIX###-scan.###DOMAIN_NAME###.
172 IN PTR ###RAC_NODE_NAME_PREFIX###-scan.###DOMAIN_NAME###.
173 IN PTR ###RAC_NODE_NAME_PREFIX###-scan.###DOMAIN_NAME###.
175 IN PTR ###RAC_NODE_NAME_PREFIX###-gns1.###DOMAIN_NAME###.
176 IN PTR ###RAC_NODE_NAME_PREFIX###-gns2.###DOMAIN_NAME###.

با اجرای دستور زیر، imageای را برای اجرای Container DNS ایجاد خواهیم کرد:

[root@OEL7 latest]# cd /OracleRepoGIT/docker-images/OracleDatabase/RAC/OracleDNSServer/dockerfiles
[root@OEL7 dockerfiles]# ./buildContainerImage.sh -v latest

==========================
DOCKER info:
Client: Debug Mode: false
Server: Containers: 0 Ruing: 0 Paused: 0 Stopped: 0 Images: 0 Server Version: 19.03.11-ol Storage Driver: overlay2 Backing Filesystem: xfs Supports d_type: true Native Overlay Diff: false Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf jouald json-file local logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: 7eba5930496d9bbe375fdf71603e610ad737d2b2 runc version: 52de29d init version: fec3683 Security Options: seccomp Profile: default Keel Version: 4.14.35-1818.3.3.el7uek.x86_64 Operating System: Oracle Linux Server 7.6 OSType: linux Architecture: x86_64 CPUs: 56 Total Memory: 755.5GiB Name: OEL7 ID: LE4E:FM37:S2RL:AUUF:CNFH:KZHE:FYT4:L6P2:EGYT:JF7S:6JWE:QAGZ Docker Root Dir: /var/lib/docker Debug Mode: false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false
Loaded plugins: ovl
Cleaning repos: ol7_latest
Removing intermediate container be8a4c33ec0e ---> 9ea5cc12a920
Step 9/11 : USER orcladmin ---> Ruing in fd4fd39557aa
Removing intermediate container fd4fd39557aa ---> 3ff83fb548b8
Step 10/11 : WORKDIR /home/orcladmin ---> Ruing in f3719e4b73f6
Removing intermediate container f3719e4b73f6 ---> 9231bfa5d4d8
Step 11/11 : CMD exec $SCRIPT_DIR/$RUN_FILE ---> Ruing in 8ee8f356b32a
Removing intermediate container 8ee8f356b32a ---> e8076b822446
Successfully built e8076b822446
Successfully tagged oracle/rac-dnsserver:latest Oracle Database Docker Image for Real Application Clusters (RAC) version latest is ready to be extended: --> oracle/rac-dnsserver:latest Build completed in 609 seconds.

Imageای با نام rac-dnsserver ایجاد شده است:

[root@OEL7 latest]# docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
oracle/rac-dnsserver latest e8076b822446 5 hours ago 285MB
oraclelinux 7-slim 60807cf6683b 12 days ago 135MB

از طریق این image، کانتینر racnode-dns را ایجاد می کنیم:

[root@OEL7 dockerfiles]# docker run -d --name racnode-dns
> --hostname racnode-dns
> --dns-search="example.com"
> --cap-add=SYS_ADMIN
> --network rac_pub_nw1
> --ip 10.0.20.2
> --sysctl net.ipv6.conf.all.disable_ipv6=1
> --env SETUP_DNS_CONFIG_FILES="setup_true"
> --env DOMAIN_NAME="example.com"
> --env RAC_NODE_NAME_PREFIX="racnode"
> oracle/rac-dnsserver:latest

c2374d70d683d211e70a61bdf835b8e9eecb8b72cd0c789c0129ebc503f8315b

با دستور زیر محتویات log مربوط به این container را مشاهده خواهید کرد:

[root@OEL7 dockerfiles]# docker logs -f racnode-dns

10-05-2022 10:30:41 UTC : : Creating /tmp/orod.log
10-05-2022 10:30:41 UTC : : HOSTNAME is set to racnode-dns
10-05-2022 10:30:41 UTC : : RAC_PUBLIC_SUBNET is set to 10.0.20
10-05-2022 10:30:41 UTC : : HOSTNAME_IP_LAST_DIGITS is set to 2
10-05-2022 10:30:41 UTC : : RAC_DNS_SERVER_IP is set to 10.0.20.2
10-05-2022 10:30:41 UTC : : RAC_PUBLIC_REVERSE_IP set to 20.0.10
10-05-2022 10:30:41 UTC : : Creating Directories
10-05-2022 10:30:41 UTC : : Copying files to destination dir
10-05-2022 10:30:41 UTC : : Setting up Zone file
10-05-2022 10:30:41 UTC : : Setting up reverse Zone file
10-05-2022 10:30:41 UTC : : Setting ip named configuration file
10-05-2022 10:30:41 UTC : : Setting up Resolve.conf file
10-05-2022 10:30:41 UTC : : Starting DNS Server
10-05-2022 10:30:41 UTC : : Checking DNS Server
Server: 10.0.20.2
Address: 10.0.20.2#53
Name: racnode-dns.example.com
Address: 10.0.20.2
10-05-2022 10:30:41 UTC : : DNS Server started sucessfully
10-05-2022 10:30:41 UTC : : ################################################
10-05-2022 10:30:41 UTC : : DNS Server IS READY TO USE!
10-05-2022 10:30:41 UTC : : ################################################
10-05-2022 10:30:41 UTC : : DNS Server Started Successfully
10-05-2022 10:30:41 UTC : : Setting up reverse Zone file
10-05-2022 10:30:41 UTC : : Setting ip named configuration file
10-05-2022 10:30:41 UTC : : Setting up Resolve.conf file
10-05-2022 10:30:41 UTC : : Starting DNS Server
10-05-2022 10:30:41 UTC : : Checking DNS Server
10-05-2022 10:30:41 UTC : : DNS Server started sucessfully
10-05-2022 10:30:41 UTC : : ################################################
10-05-2022 10:30:41 UTC : : DNS Server IS READY TO USE!
10-05-2022 10:30:41 UTC : : ################################################
10-05-2022 10:30:41 UTC : : DNS Server Started Successfully

با دستور زیر اطلاعات بیشتری از این container مشاهده می کنید:

[root@OEL7 ~]# docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c2374d70d683 oracle/rac-dnsserver:latest "/bin/sh -c 'exec $S…" 5 hours ago Up 5 hours racnode-dns

برای آنکه ببینیم DNS به خوبی عمل می کند، به container وصل شده و از دستور nslookup کمک می گیریم:

[root@OEL7 ~]# docker exec -it racnode-dns /bin/bash
[orcladmin@racnode-dns ~]$ nslookup racnode1
Server: 10.0.20.2
Address: 10.0.20.2#53
Name: racnode1.example.com
Address: 10.0.20.151
[orcladmin@racnode-dns ~]$ nslookup racnode2
Server: 10.0.20.2
Address: 10.0.20.2#53
Name: racnode2.example.com
Address: 10.0.20.152
[orcladmin@racnode-dns ~]$ nslookup 10.0.20.152
152.20.0.10.in-addr.arpa name = racnode2.example.com.

همچنین برای کلاستری که قرار است نصب شود، سه scan ip در نظر گرفته شده است:

[orcladmin@racnode-dns ~]$ nslookup racnode-scan.example.com.
Server: 10.0.20.2
Address: 10.0.20.2#53
Name: racnode-scan.example.com
Address: 10.0.20.173
Name: racnode-scan.example.com
Address: 10.0.20.172
Name: racnode-scan.example.com
Address: 10.0.20.171
[orcladmin@racnode-dns ~]$ nslookup 10.0.20.171
171.20.0.10.in-addr.arpa name = racnode-scan.example.com.

ایجاد container برای راه اندازی نود اول کلاستر

در این مرحله قصد داریم docker image مربوط به Oracle RAC را ایجاد کنیم برای این کار، فایلهای نصب grid و database را در مسیر زیر کپی می کنیم:

[root@OEL7 dockerfiles]# cd /OracleRepoGIT/docker-images/OracleDatabase/RAC/OracleRealApplicationClusters/dockerfiles/21.3.0/
[root@OEL7 21.3.0]# cp /source/LINUX.X64_213000_* .

بعد از انجام کپی، با اجرای دستور زیر، image مربوط به کلاستر را ایجاد می کنیم:

[root@OEL7 ~]# cd /OracleRepoGIT/docker-images/OracleDatabase/RAC/OracleRealApplicationClusters/dockerfiles
[root@OEL7 dockerfiles]# ./buildContainerImage.sh -v 21.3.0

Checking if required packages are present and valid...
LINUX.X64_213000_db_home.zip: OK
LINUX.X64_213000_grid_home.zip: OK
==========================
DOCKER info:
Client: Debug Mode: false
Server: Containers: 1 Ruing: 1 Paused: 0 Stopped: 0 Images: 25 Server Version: 19.03.11-ol Storage Driver: overlay2 Backing Filesystem: xfs Supports d_type: true Native Overlay Diff: false Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf jouald json-file local logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: 7eba5930496d9bbe375fdf71603e610ad737d2b2 runc version: 52de29d init version: fec3683 Security Options: seccomp Profile: default Keel Version: 4.14.35-1818.3.3.el7uek.x86_64 Operating System: Oracle Linux Server 7.6 OSType: linux Architecture: x86_64 CPUs: 56 Total Memory: 755.5GiB Name: OEL7 ID: LE4E:FM37:S2RL:AUUF:CNFH:KZHE:FYT4:L6P2:EGYT:JF7S:6JWE:QAGZ Docker Root Dir: /var/lib/docker Debug Mode: false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false
Registries:
==========================
Building image 'oracle/database-rac:21.3.0' ...
Sending build context to Docker daemon 5.532GB
Step 1/26 : ARG BASE_OL_IMAGE=oraclelinux:7-slim
Step 2/26 : FROM $BASE_OL_IMAGE AS base ---> 60807cf6683b
Step 3/26 : LABEL "provider"="Oracle" "issues"="https://github.com/oracle/docker-images/issues" "volume.setup.location1"="/opt/scripts" "volume.startup.location1"="/opt/scripts/startup" "port.listener"="1521" "port.oemexpress"="5500" ---> Ruing in 38596d9a815a
……
---> Ruing in fbb006aefde1
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
Check /u01/app/21.3.0/grid/install/root_fbb006aefde1_2022-10-05_11-13-31-659210429.log for the output of root script
Check /u01/app/oracle/product/21.3.0/dbhome_1/install/root_fbb006aefde1_2022-10-05_11-13-31-779049670.log for the output of root script
Preparing... ########################################
Updating / installing...
cvuqdisk-1.0.10-1 ########################################
Removing intermediate container fbb006aefde1 ---> 98380a74fcf1
Step 23/26 : USER ${USER} ---> Ruing in 3d39de0dd47e
Removing intermediate container 3d39de0dd47e ---> c03fe8661c27
Step 24/26 : VOLUME ["/common_scripts"] ---> Ruing in ae9f2de3b766
Removing intermediate container ae9f2de3b766 ---> dc3628ed0345
Step 25/26 : WORKDIR $WORKDIR ---> Ruing in 511736d2dc3a
Removing intermediate container 511736d2dc3a ---> 6bd0ac840e4b
Step 26/26 : ENTRYPOINT /usr/bin/$INITSH ---> Ruing in 1376bcd74cfb
Removing intermediate container 1376bcd74cfb ---> 023775e51a76
Successfully built 023775e51a76
Successfully tagged oracle/database-rac:21.3.0 Oracle Database Docker Image for Real Application Clusters (RAC) version 21.3.0 is ready to be extended: --> oracle/database-rac:21.3.0 Build completed in 1513 seconds.

Image مربوط به oracle RAC ایجاد شده و آماده اجرا می باشد:

[root@OEL7 ~]# docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
oracle/database-rac 21.3.0 023775e51a76 5 hours ago 14.1GB
<none> <none> 0226d841eb6d 5 hours ago 19.7GB
<none> <none> 9936f433aeb4 5 hours ago 5.93GB
oracle/rac-dnsserver latest e8076b822446 5 hours ago 285MB
oraclelinux 7-slim 60807cf6683b 12 days ago 135MB

قبل از اجرای container از image ایجاد شده باید فایلهای زیر را در داکر هاست ایجاد کنیم تا در دسترس همه containerها باشند.

اولین فایل rac_host_file است که قرار است فایل hosts را(برای name resolution) بین همه containerها به اشتراک بگذارد:

[root@OEL7 ~]# mkdir /opt/containers; touch /opt/containers/rac_host_file

همانطور که می دانید، در زمان نصب grid infra و نرم افزار oracle باید برای کاربر oracle و grid پسوردی را تنظیم کرده باشیم و این مسئله برای کاربران دیتابیسی هم صادق است. برای این کار، با اجرای دستورات زیر، فایل پسوردی را در داکر هاست ایجاد می کنیم تا در طول راه اندازی کلاستر، از این پسورد استفاده شود:

[root@OEL7 ~]# mkdir /opt/.secrets/
[root@OEL7 ~]# openssl rand -hex 64 -out /opt/.secrets/pwd.key
[root@OEL7 ~]# echo "Welcome1" > /opt/.secrets/common_os_pwdfile
[root@OEL7 ~]# openssl enc -aes-256-cbc -salt -in /opt/.secrets/common_os_pwdfile -out /opt/.secrets/common_os_pwdfile.enc -pass file:/opt/.secrets/pwd.key
[root@OEL7 ~]# rm -f /opt/.secrets/common_os_pwdfile
[root@OEL7 ~]# chmod 400 /opt/.secrets/common_os_pwdfile.enc; chmod 400 /opt/.secrets/pwd.key

همانطور که می دانید، برای نصب Oracle RAC نیاز به فضای مشترک بین دو نود داریم، اگر دیسک و یا پارتیشن مستقلی برای این کار در هاست دارید، نیازی به انجام این مرحله نیست در غیر این صورت می توانید با دستور dd فایلی را ایجاد کنید تا نقش این دیسک را برای شما بازی کند:

[root@OEL7 ~]# dd if=/dev/zero of=/oracle/ASM1 bs=1024 count=50388608
50388608+0 records in
50388608+0 records out
51597934592 bytes (52 GB) copied, 137.139 s, 376 MB/s
[root@OEL7 ~]# dd if=/dev/zero of=/oracle/ASM2 bs=1024 count=50388608
50388608+0 records in
50388608+0 records out
51597934592 bytes (52 GB) copied, 142.893 s, 361 MB/s
[root@OEL7 ~]# losetup /dev/loop0 /oracle/ASM1
[root@OEL7 ~]# losetup /dev/loop1 /oracle/ASM2

–ایجاد container

با اجرای دستور زیر، containerای با نام racnode1 ایجاد خواهیم کرد که قرار است نود اول کلاستر در این container ایجاد و اجرا شود:

[root@OEL7 ~]# docker create -t -i --volume /dev/shm
> --hostname racnode1
> --volume /boot:/boot:ro
> --volume /dev/shm --volume /opt/containers/rac_host_file:/etc/hosts --volume /opt/.secrets:/run/secrets:ro
> --tmpfs /dev/shm:rw,exec,size=4G --cpuset-cpus 0-3
> --volume /opt/containers/rac_host_file:/etc/hosts
> --volume /opt/.secrets:/run/secrets:ro
> --volume /etc/localtime:/etc/localtime:ro
> --cpuset-cpus 0-3 --dns-search=example.com
> --memory 16G
> --memory-swap 32G
> --sysctl keel.shmall=2097152
> --sysctl "keel.sem=250 32000 100 128"
> --sysctl keel.shmmax=8589934592
> --sysctl keel.shmmni=4096
> --dns-search=example.com
> --dns=10.0.20.2
> --device=/dev/loop0:/dev/asm_disk1 -e VIP_HOSTNAME=racnode1-vip -e PRIV_IP=192.168.10.151
> --device=/dev/loop1:/dev/asm_disk2
> --privileged=false
> --cap-add=SYS_NICE
> --cap-add=SYS_RESOURCE
> --cap-add=NET_ADMIN
> -e DNS_SERVERS=10.0.20.2
> -e NODE_VIP=10.0.20.161
> -e VIP_HOSTNAME=racnode1-vip
> -e PRIV_IP=192.168.10.151
> -e PRIV_HOSTNAME=racnode1-priv
> -e PUBLIC_IP=10.0.20.151
> -e PUBLIC_HOSTNAME=racnode1
> -e SCAN_NAME=racnode-scan
> -e OP_TYPE=INSTALL
> -e DOMAIN=example.com
> -e ASM_DEVICE_LIST=/dev/asm_disk1,/dev/asm_disk2
> -e ASM_DISCOVERY_DIR=/dev
> -e COMMON_OS_PWD_FILE=common_os_pwdfile.enc
> -e PWD_KEY=pwd.key
> --restart=always --tmpfs=/run -v /sys/fs/cgroup:/sys/fs/cgroup:ro
> --cpu-rt-runtime=95000 --ulimit rtprio=99
> --name racnode1
> oracle/database-rac:21.3.0

2b66bf78fe1eecb1868f29e29b89eddee836496363e8df41c7377fa5ecf9b736

قبل از استارت این container، آن را به دو شبکه ایجاد شده متصل می کنیم:

[root@OEL7 ~]# docker network discoect bridge racnode1
[root@OEL7 ~]# docker network coect rac_pub_nw1 --ip 10.0.20.151 racnode1
[root@OEL7 ~]# docker network coect rac_priv_nw1 --ip 192.168.10.151 racnode1

بعد از این تنظیمات، container را استارت می کنیم:

[root@OEL7 backup]# docker start racnode1
racnode1
[root@OEL7 backup]# docker logs -f racnode1

Creating env variables file /etc/rac_env_vars
Starting Systemd
systemd 219 ruing in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
Detected virtualization other.
Detected architecture x86-64.
Welcome to Oracle Linux Server 7.9!
Set hostname to <racnode1>.
[/usr/lib/systemd/system/systemd-pstore.service:22] Unknown lvalue 'StateDirectory' in section 'Service'
Caot add dependency job for unit display-manager.service, ignoring: Unit not found.
[ OK ] Started Forward Password Requests to Wall Directory Watch.
[ OK ] Reached target Local Encrypted Volumes.
[ OK ] Started Dispatch Password Requests to Console Directory Watch.
[ OK ] Reached target RPC Port Mapper.
[ OK ] Created slice Root Slice.
[ OK ] Created slice User and Session Slice.
[ OK ] Created slice System Slice.
[ OK ] Created slice system-getty.slice.
[ OK ] Reached target Slices.
[ OK ] Listening on Delayed Shutdown Socket.
[ OK ] Reached target Swap.
[ OK ] Listening on Joual Socket. Starting Rebuild Hardware Database...
[ OK ] Reached target Local File Systems (Pre). Starting Joual Service... Starting Read and set NIS domaiame from /etc/sysconfig/network... Starting Configure read-only root support...
[ OK ] Listening on /dev/initctl Compatibility Named Pipe.
[ OK ] Started Read and set NIS domaiame from /etc/sysconfig/network.
[ OK ] Started Joual Service. Starting Flush Joual to Persistent Storage...
[ OK ] Started Configure read-only root support.
[ OK ] Reached target Local File Systems. Starting Rebuild Joual Catalog... Starting Preprocess NFS configuration... Starting Mark the need to relabel after reboot... Starting Load/Save Random Seed...
[ OK ] Started Flush Joual to Persistent Storage.
[ OK ] Started Mark the need to relabel after reboot. Starting Create Volatile Files and Directories...
[ OK ] Started Rebuild Joual Catalog.
[ OK ] Started Load/Save Random Seed.
[ OK ] Started Preprocess NFS configuration.
[ OK ] Started Create Volatile Files and Directories. Mounting RPC Pipe File System... Starting Update UTMP about System Boot/Shutdown...
[FAILED] Failed to mount RPC Pipe File System.
See 'systemctl status var-lib-nfs-rpc_pipefs.mount' for details.
[DEPEND] Dependency failed for rpc_pipefs.target.
[DEPEND] Dependency failed for RPC security service for NFS client and server.
[ OK ] Started Update UTMP about System Boot/Shutdown.
[ OK ] Started Rebuild Hardware Database. Starting Update is Completed...
[ OK ] Started Update is Completed.
[ OK ] Reached target System Initialization.
[ OK ] Started Flexible branding.
[ OK ] Reached target Paths.
[ OK ] Listening on RPCbind Server Activation Socket. Starting RPC bind service...
[ OK ] Started Daily Cleanup of Temporary Directories.
[ OK ] Reached target Timers.
[ OK ] Listening on D-Bus System Message Bus Socket.
[ OK ] Reached target Sockets.
[ OK ] Reached target Basic System. Starting OpenSSH Server Key Generation... Starting Resets System Activity Logs... Starting GSSAPI Proxy Daemon...
[ OK ] Started D-Bus System Message Bus. Starting Self Monitoring and Reporting Technology (SMART) Daemon... Starting LSB: Bring up/down networking... Starting Login Service...
[ OK ] Started RPC bind service.
[ OK ] Started Resets System Activity Logs.
[ OK ] Started GSSAPI Proxy Daemon. Starting Cleanup of Temporary Directories...
[ OK ] Reached target NFS client services.
[ OK ] Reached target Remote File Systems (Pre).
[ OK ] Reached target Remote File Systems. Starting Permit User Sessions...
[ OK ] Started Cleanup of Temporary Directories.
[ OK ] Started Permit User Sessions.
[ OK ] Started Command Scheduler.
[ OK ] Started Login Service.
[ OK ] Started OpenSSH Server Key Generation.
[ OK ] Started LSB: Bring up/down networking.
[ OK ] Reached target Network.
[ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart... Starting OpenSSH server daemon... Starting /etc/rc.d/rc.local Compatibility...
[ OK ] Started Notify NFS peers of a restart.
[ OK ] Started /etc/rc.d/rc.local Compatibility.
[ OK ] Started Console Getty.
[ OK ] Reached target Login Prompts.
[ OK ] Started OpenSSH server daemon.
10-05-2022 14:50:53 +0330 : : Process id of the program :
10-05-2022 14:50:53 +0330 : : #################################################
10-05-2022 14:50:53 +0330 : : Starting Grid Installation
10-05-2022 14:50:53 +0330 : : #################################################
10-05-2022 14:50:53 +0330 : : Pre-Grid Setup steps are in process
10-05-2022 14:50:53 +0330 : : Process id of the program :
10-05-2022 14:50:53 +0330 : : Disable failed service var-lib-nfs-rpc_pipefs.mount
10-05-2022 14:50:53 +0330 : : Resetting Failed Services
10-05-2022 14:50:53 +0330 : : Sleeping for 60 seconds
[ OK ] Started Self Monitoring and Reporting Technology (SMART) Daemon.
[ OK ] Reached target Multi-User System.
[ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes...
[ OK ] Started Update UTMP about System Runlevel Changes.
Oracle Linux Server 7.9
Keel 4.14.35-1818.3.3.el7uek.x86_64 on an x86_64
racnode1 login: 10-05-2022 14:51:53 +0330 : : Systemctl state is ruing!
10-05-2022 14:51:53 +0330 : : Setting correct permissions for /bin/ping
10-05-2022 14:51:53 +0330 : : Public IP is set to 10.0.20.151
10-05-2022 14:51:53 +0330 : : RAC Node PUBLIC Hostname is set to racnode1
10-05-2022 14:51:53 +0330 : : Preparing host line for racnode1
10-05-2022 14:51:53 +0330 : : Adding n10.0.20.151tracnode1.example.comtracnode1 to /etc/hosts
10-05-2022 14:51:53 +0330 : : Preparing host line for racnode1-priv
10-05-2022 14:51:53 +0330 : : Adding n192.168.10.151tracnode1-priv.example.comtracnode1-priv to /etc/hosts
10-05-2022 14:51:53 +0330 : : Preparing host line for racnode1-vip
10-05-2022 14:51:53 +0330 : : Adding n10.0.20.161tracnode1-vip.example.comtracnode1-vip to /etc/hosts
10-05-2022 14:51:53 +0330 : : Preparing host line for racnode-scan
10-05-2022 14:51:53 +0330 : : Preapring Device list
10-05-2022 14:51:53 +0330 : : Changing Disk permission and ownership /dev/asm_disk1
10-05-2022 14:51:53 +0330 : : Changing Disk permission and ownership /dev/asm_disk2
10-05-2022 14:51:53 +0330 : : Preapring Dns Servers list
10-05-2022 14:51:53 +0330 : : Setting DNS Servers
10-05-2022 14:51:53 +0330 : : Adding nameserver 10.0.20.2 in /etc/resolv.conf.
10-05-2022 14:51:53 +0330 : : #####################################################################
10-05-2022 14:51:53 +0330 : : RAC setup will begin in 2 minutes
10-05-2022 14:51:53 +0330 : : ####################################################################
10-05-2022 14:52:23 +0330 : : ###################################################
10-05-2022 14:52:23 +0330 : : Pre-Grid Setup steps completed
10-05-2022 14:52:23 +0330 : : ###################################################
10-05-2022 14:52:23 +0330 : : Checking if grid is already configured
10-05-2022 14:52:23 +0330 : : Process id of the program :
10-05-2022 14:52:23 +0330 : : Public IP is set to 10.0.20.151
10-05-2022 14:52:23 +0330 : : RAC Node PUBLIC Hostname is set to racnode1
10-05-2022 14:52:23 +0330 : : Domain is defined to example.com
10-05-2022 14:52:23 +0330 : : Default setting of AUTO GNS VIP set to false. If you want to use AUTO GNS VIP, please pass DHCP_CONF as an env parameter set to true
10-05-2022 14:52:23 +0330 : : RAC VIP set to 10.0.20.161
10-05-2022 14:52:23 +0330 : : RAC Node VIP hostname is set to racnode1-vip
10-05-2022 14:52:23 +0330 : : SCAN_NAME name is racnode-scan
10-05-2022 14:52:23 +0330 : : SCAN PORT is set to empty string. Setting it to 1521 port.
10-05-2022 14:52:24 +0330 : : 10.0.20.171
10.0.20.173
10.0.20.172
10-05-2022 14:52:24 +0330 : : SCAN Name resolving to IP. Check Passed!
10-05-2022 14:52:24 +0330 : : SCAN_IP set to the empty string
10-05-2022 14:52:24 +0330 : : RAC Node PRIV IP is set to 192.168.10.151
10-05-2022 14:52:24 +0330 : : RAC Node private hostname is set to racnode1-priv
10-05-2022 14:52:24 +0330 : : CMAN_NAME set to the empty string
10-05-2022 14:52:24 +0330 : : CMAN_IP set to the empty string
10-05-2022 14:52:24 +0330 : : Cluster Name is not defined
10-05-2022 14:52:24 +0330 : : Cluster name is set to 'racnode-c'
10-05-2022 14:52:24 +0330 : : Password file generated
10-05-2022 14:52:24 +0330 : : Common OS Password string is set for Grid user
10-05-2022 14:52:24 +0330 : : Common OS Password string is set for Oracle user
10-05-2022 14:52:24 +0330 : : Common OS Password string is set for Oracle Database
10-05-2022 14:52:24 +0330 : : Setting CONFIGURE_GNS to false
10-05-2022 14:52:24 +0330 : : GRID_RESPONSE_FILE env variable set to empty. configGrid.sh will use standard cluster responsefile
10-05-2022 14:52:24 +0330 : : Location for User script SCRIPT_ROOT set to /common_scripts
10-05-2022 14:52:24 +0330 : : IGNORE_CVU_CHECKS is set to true
10-05-2022 14:52:24 +0330 : : Oracle SID is set to ORCLCDB
10-05-2022 14:52:24 +0330 : : Oracle PDB name is set to ORCLPDB
10-05-2022 14:52:24 +0330 : : Check passed for network card eth1 for public IP 10.0.20.151
10-05-2022 14:52:24 +0330 : : Public Netmask : 255.255.255.0
10-05-2022 14:52:24 +0330 : : Check passed for network card eth0 for private IP 192.168.10.151
10-05-2022 14:52:24 +0330 : : Building NETWORK_STRING to set networkInterfaceList in Grid Response File
10-05-2022 14:52:24 +0330 : : Network InterfaceList set to eth1:10.0.20.0:1,eth0:192.168.10.0:5
10-05-2022 14:52:24 +0330 : : Setting random password for grid user
10-05-2022 14:52:24 +0330 : : Setting random password for oracle user
10-05-2022 14:52:24 +0330 : : Calling setupSSH function
10-05-2022 14:52:24 +0330 : : SSh will be setup among racnode1 nodes
10-05-2022 14:52:24 +0330 : : Ruing SSH setup for grid user between nodes racnode1
10-05-2022 14:53:00 +0330 : : Ruing SSH setup for oracle user between nodes racnode1
10-05-2022 14:53:05 +0330 : : SSH check fine for the racnode1
10-05-2022 14:53:06 +0330 : : SSH check fine for the oracle@racnode1
10-05-2022 14:53:06 +0330 : : Preapring Device list
10-05-2022 14:53:06 +0330 : : Changing Disk permission and ownership
10-05-2022 14:53:06 +0330 : : Changing Disk permission and ownership
10-05-2022 14:53:06 +0330 : : ASM Disk size : 0
10-05-2022 14:53:06 +0330 : : ASM Device list will be with failure groups /dev/asm_disk1,,/dev/asm_disk2,
10-05-2022 14:53:06 +0330 : : ASM Device list will be groups /dev/asm_disk1,/dev/asm_disk2
10-05-2022 14:53:06 +0330 : : CLUSTER_TYPE env variable is set to STANDALONE, will not process GIMR DEVICE list as default Diskgroup is set to DATA. GIMR DEVICE List will be processed when CLUSTER_TYPE is set to DOMAIN for DSC
10-05-2022 14:53:06 +0330 : : Nodes in the cluster racnode1
10-05-2022 14:53:06 +0330 : : Setting Device permissions for RAC Install on racnode1
10-05-2022 14:53:06 +0330 : : Preapring ASM Device list
10-05-2022 14:53:06 +0330 : : Changing Disk permission and ownership
10-05-2022 14:53:06 +0330 : : Command : su - $GRID_USER -c "ssh $node sudo chown $GRID_USER:asmadmin $device" execute on racnode1
10-05-2022 14:53:06 +0330 : : Command : su - $GRID_USER -c "ssh $node sudo chmod 660 $device" execute on racnode1
10-05-2022 14:53:06 +0330 : : Populate Rac Env Vars on Remote Hosts
10-05-2022 14:53:06 +0330 : : Command : su - $GRID_USER -c "ssh $node sudo echo "export ASM_DEVICE_LIST=${ASM_DEVICE_LIST}" >> /etc/rac_env_vars" execute on racnode1
10-05-2022 14:53:06 +0330 : : Changing Disk permission and ownership
10-05-2022 14:53:06 +0330 : : Command : su - $GRID_USER -c "ssh $node sudo chown $GRID_USER:asmadmin $device" execute on racnode1
10-05-2022 14:53:06 +0330 : : Command : su - $GRID_USER -c "ssh $node sudo chmod 660 $device" execute on racnode1
10-05-2022 14:53:06 +0330 : : Populate Rac Env Vars on Remote Hosts
10-05-2022 14:53:06 +0330 : : Command : su - $GRID_USER -c "ssh $node sudo echo "export ASM_DEVICE_LIST=${ASM_DEVICE_LIST}" >> /etc/rac_env_vars" execute on racnode1
10-05-2022 14:53:06 +0330 : : Generating Reponsefile
10-05-2022 14:53:06 +0330 : : Ruing cluvfy Checks
10-05-2022 14:53:06 +0330 : : Performing Cluvfy Checks
10-05-2022 14:53:45 +0330 : : Checking /tmp/cluvfy_check.txt if there is any failed check.
This standalone version of CVU is "454" days old. The latest release of standalone CVU can be obtained from the Oracle support site. Refer to MOS note 2731675.1 for more details.
Performing following verification checks ... Physical Memory ...PASSED Available Physical Memory ...PASSED Swap Size ...PASSED Free Space: racnode1:/usr,racnode1:/var,racnode1:/etc,racnode1:/sbin,racnode1:/tmp ...PASSED User Existence: grid ... Users With Same UID: 54332 ...PASSED User Existence: grid ...PASSED Group Existence: asmadmin ...PASSED Group Existence: asmdba ...PASSED Group Existence: oinstall ...PASSED Group Membership: asmdba ...PASSED Group Membership: asmadmin ...PASSED Group Membership: oinstall(Primary) ...PASSED Run Level ...PASSED Hard Limit: maximum open file descriptors ...PASSED Soft Limit: maximum open file descriptors ...PASSED Hard Limit: maximum user processes ...PASSED Soft Limit: maximum user processes ...PASSED Soft Limit: maximum stack size ...PASSED Architecture ...PASSED OS Keel Version ...PASSED OS Keel Parameter: semmsl ...PASSED OS Keel Parameter: semmns ...PASSED OS Keel Parameter: semopm ...PASSED OS Keel Parameter: semmni ...PASSED OS Keel Parameter: shmmax ...PASSED OS Keel Parameter: shmmni ...PASSED OS Keel Parameter: shmall ...PASSED OS Keel Parameter: file-max ...PASSED OS Keel Parameter: ip_local_port_range ...PASSED OS Keel Parameter: rmem_default ...PASSED OS Keel Parameter: rmem_max ...PASSED OS Keel Parameter: wmem_default ...PASSED OS Keel Parameter: wmem_max ...PASSED OS Keel Parameter: aio-max-nr ...PASSED OS Keel Parameter: panic_on_oops ...PASSED Package: kmod-20-21 (x86_64) ...PASSED Package: kmod-libs-20-21 (x86_64) ...PASSED Package: binutils-2.23.52.0.1 ...PASSED Package: libgcc-4.8.2 (x86_64) ...PASSED Package: libstdc++-4.8.2 (x86_64) ...PASSED Package: sysstat-10.1.5 ...PASSED Package: ksh ...PASSED Package: make-3.82 ...PASSED Package: glibc-2.17 (x86_64) ...PASSED Package: glibc-devel-2.17 (x86_64) ...PASSED Package: libaio-0.3.109 (x86_64) ...PASSED Package: nfs-utils-1.2.3-15 ...PASSED Package: smartmontools-6.2-4 ...PASSED Package: net-tools-2.0-0.17 ...PASSED Package: policycoreutils-2.5-17 ...PASSED Package: policycoreutils-python-2.5-17 ...PASSED Port Availability for component "Oracle Remote Method Invocation (ORMI)" ...PASSED Port Availability for component "Oracle Notification Service (ONS)" ...PASSED Port Availability for component "Oracle Cluster Synchronization Services (CSSD)" ...PASSED Port Availability for component "Oracle Notification Service (ONS) Enterprise Manager support" ...PASSED Port Availability for component "Oracle Database Listener" ...PASSED Users With Same UID: 0 ...PASSED Current Group ID ...PASSED Root user consistency ...PASSED Host name ...PASSED Node Coectivity ... Hosts File ...PASSED Check that maximum (MTU) size packet goes through subnet ...PASSED Node Coectivity ...PASSED Multicast or broadcast check ...PASSED ASM Network ...PASSED Device Checks for ASM ... Access Control List check ...PASSED Device Checks for ASM ...PASSED Same core file name patte ...PASSED User Mask ...PASSED User Not In Group "root": grid ...PASSED Time zone consistency ...PASSED Path existence, ownership, permissions and attributes ... Path "/var" ...PASSED Path "/var/lib/oracle" ...PASSED Path "/dev/shm" ...PASSED Path existence, ownership, permissions and attributes ...PASSED VIP Subnet configuration check ...PASSED resolv.conf Integrity ...PASSED DNS/NIS name service ... Name Service Switch Configuration File Integrity ...PASSED DNS/NIS name service ...PASSED Single Client Access Name (SCAN) ...PASSED Domain Sockets ...PASSED Daemon "avahi-daemon" not configured and ruing ...PASSED Daemon "proxyt" not configured and ruing ...PASSED loopback network interface address ...PASSED Oracle base: /u01/app/grid ... '/u01/app/grid' ...PASSED Oracle base: /u01/app/grid ...PASSED User Equivalence ...PASSED RPM Package Manager database ...INFORMATION (PRVG-11250) Network interface bonding status of private intercoect network interfaces ...PASSED /dev/shm mounted as temporary file system ...PASSED File system mount options for path /var ...PASSED DefaultTasksMax parameter ...PASSED zeroconf check ...PASSED ASM Filter Driver configuration ...PASSED Systemd login manager IPC parameter ...PASSED Systemd status ...PASSED
Pre-check for cluster services setup was successful.
RPM Package Manager database ...INFORMATION
PRVG-11250 : The check "RPM Package Manager database" was not performed because
it needs 'root' user privileges.
Refer to My Oracle Support notes "2548970.1" for more details regarding errors
PRVG-11250".
CVU operation performed: stage -pre crsinst
Date: Oct 5, 2022 11:23:07 AM
CVU home: /u01/app/21.3.0/grid
User: grid
Operating system: Linux4.14.35-1818.3.3.el7uek.x86_64
10-05-2022 14:53:45 +0330 : : CVU Checks are ignored as IGNORE_CVU_CHECKS set to true. It is recommended to set IGNORE_CVU_CHECKS to false and meet all the cvu checks requirement. RAC installation might fail, if there are failed cvu checks.
10-05-2022 14:53:45 +0330 : : Ruing Grid Installation
10-05-2022 14:54:38 +0330 : : Ruing root.sh
10-05-2022 14:54:38 +0330 : : Nodes in the cluster racnode1
10-05-2022 14:54:38 +0330 : : Ruing root.sh on racnode1
10-05-2022 15:02:34 +0330 : : Ruing post root.sh steps
10-05-2022 15:02:34 +0330 : : Ruing post root.sh steps to setup Grid env
10-05-2022 15:03:32 +0330 : : Checking Cluster Status
10-05-2022 15:03:32 +0330 : : Nodes in the cluster
10-05-2022 15:03:32 +0330 : : Removing /tmp/cluvfy_check.txt as cluster check has passed
10-05-2022 15:03:32 +0330 : : Generating DB Responsefile Ruing DB creation
10-05-2022 15:03:32 +0330 : : Ruing DB creation
10-05-2022 15:03:33 +0330 : : Workaround for Bug 32449232 : Removing /u01/app/grid/kfod
10-05-2022 15:28:54 +0330 : : Checking DB status
10-05-2022 15:28:55 +0330 : : #################################################################
10-05-2022 15:28:55 +0330 : : Oracle Database ORCLCDB is up and ruing on racnode1
10-05-2022 15:28:55 +0330 : : #################################################################
10-05-2022 15:28:55 +0330 : : Ruing User Script
10-05-2022 15:28:55 +0330 : : Setting Remote Listener
10-05-2022 15:28:55 +0330 : : ####################################
10-05-2022 15:28:55 +0330 : : ORACLE RAC DATABASE IS READY TO USE!
10-05-2022 15:28:55 +0330 : : ####################################

کلاستر با یک نود در کانتینر racnode1 در حال اجرا است:

grid 20492 1 0 14:59 ? 00:00:00 asm_pmon_+ASM1
oracle 51242 1 0 15:10 ? 00:00:00 ora_pmon_ORCLCDB1

[grid@racnode1 ~]$ crsctl stat res -t

--------------------------------------------------------------------- -----------
Name Target State Server State de tails
--------------------------------------------------------------------- -----------
Local Resources
--------------------------------------------------------------------- -----------
ora.LISTENER.lsnr ONLINE ONLINE racnode1 STABLE
ora.chad ONLINE ONLINE racnode1 STABLE
ora.net1.network ONLINE ONLINE racnode1 STABLE
ora.ons ONLINE ONLINE racnode1 STABLE
--------------------------------------------------------------------- -----------
Cluster Resources
--------------------------------------------------------------------- -----------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup) 1 ONLINE ONLINE racnode1 STABLE
ora.DATA.dg(ora.asmgroup) 1 ONLINE ONLINE racnode1 STABLE
ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE racnode1 STABLE
ora.LISTENER_SCAN2.lsnr 1 ONLINE ONLINE racnode1 STABLE
ora.LISTENER_SCAN3.lsnr 1 ONLINE ONLINE racnode1 STABLE
ora.asm(ora.asmgroup) 1 ONLINE ONLINE racnode1 Started, STABLE
ora.asmnet1.asmnetwork(ora.asmgroup) 1 ONLINE ONLINE racnode1 STABLE
ora.cdp1.cdp 1 ONLINE ONLINE racnode1 STABLE
ora.cdp2.cdp 1 ONLINE ONLINE racnode1 STABLE
ora.cdp3.cdp 1 ONLINE ONLINE racnode1 STABLE
ora.cvu 1 ONLINE ONLINE racnode1 STABLE
ora.orclcdb.db 1 ONLINE ONLINE racnode1 Open,HOME=/u01/app/oracle/product/21.3.0/dbhome_1,STABLE
ora.orclcdb.orclpdb.pdb 1 ONLINE ONLINE racnode1 STABLE
ora.qosmserver 1 ONLINE ONLINE racnode1 STABLE
ora.racnode1.vip 1 ONLINE ONLINE racnode1 STABLE
ora.scan1.vip 1 ONLINE ONLINE racnode1 STABLE
ora.scan2.vip 1 ONLINE ONLINE racnode1 STABLE
ora.scan3.vip 1 ONLINE ONLINE racnode1 STABLE

همانطور که می بینید، دیتابیس، orclcdb هم با pdbای به نام orclpdb ایجاد شده است:

[oracle@racnode1 ~]$ export ORACLE_HOME=/u01/app/oracle/product/21.3.0/dbhome_1
[oracle@racnode1 ~]$ export ORACLE_SID=ORCLCDB1
[oracle@racnode1 ~]$ sqlplus "/as sysdba"

SQL*Plus: Release 21.0.0.0.0 - Production on Wed Oct 5 19:53:46 2022
Version 21.3.0.0.0
Copyright (c) 1982, 2021, Oracle. All rights reserved.
Coected to:
Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production
Version 21.3.0.0.0
SQL> show parameter sga
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
sga_max_size big integer 3760M
sga_target big integer 3760M
SQL> show pdbs CON_ID CON_NAME OPEN MODE RESTRICTED
---------- ------------------------------ ---------- ---------- 2 PDB$SEED READ ONLY NO 3 ORCLPDB READ WRITE NO
SQL> show parameter cluster_data
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
cluster_database boolean TRUE

اضافه کردن نود دوم

مراحل انجام شده برای ایجاد container نود اول را در این قسمت برای اصافه کردن نود دوم تکرار می کنیم. این کار در پس زمینه با اجرای اسکریپت addnode انجام خواهد شد:

[root@OEL7 ~]# docker create -t -i --volume /dev/shm
> --hostname racnode2 --tmpfs /dev/shm:rw,exec,size=4G
> --volume /boot:/boot:ro
> --volume /dev/shm
> --tmpfs /dev/shm:rw,exec,size=4G
> --volume /opt/containers/rac_host_file:/etc/hosts
> --volume /opt/.secrets:/run/secrets:ro
> --volume /etc/localtime:/etc/localtime:ro
> --cpuset-cpus 4-7
> --memory 16G
> --memory-swap 32G
> --sysctl keel.shmall=2097152
> --sysctl "keel.sem=250 32000 100 128"
> --sysctl keel.shmmax=8589934592
> --sysctl keel.shmmni=4096
> --dns-search=example.com
> --dns=10.0.20.2
> --device=/dev/loop0:/dev/asm_disk1
> --device=/dev/loop1:/dev/asm_disk2
> --privileged=false -e DNS_SERVERS=10.0.20.2
> --cap-add=SYS_NICE
> --cap-add=SYS_RESOURCE
> --cap-add=NET_ADMIN -e PRIV_IP=192.168.10.152 -e PRIV_HOSTNAME=racnode2-priv
> -e EXISTING_CLS_NODES=racnode1
> -e DNS_SERVERS=10.0.20.2
> -e NODE_VIP=10.0.20.162
> -e VIP_HOSTNAME=racnode2-vip
> -e PRIV_IP=192.168.10.152
> -e PRIV_HOSTNAME=racnode2-priv
> -e PUBLIC_IP=10.0.20.152
> -e PUBLIC_HOSTNAME=racnode2
> -e SCAN_NAME=racnode-scan
> -e OP_TYPE=ADDNODE
> -e DOMAIN=example.com
> -e ASM_DEVICE_LIST=/dev/asm_disk1,/dev/asm_disk2
> -e ASM_DISCOVERY_DIR=/dev
> -e ORACLE_SID=ORCLCDB
> -e COMMON_OS_PWD_FILE=common_os_pwdfile.enc
> -e PWD_KEY=pwd.key
> --restart=always --tmpfs=/run -v /sys/fs/cgroup:/sys/fs/cgroup:ro
> --cpu-rt-runtime=95000 --ulimit rtprio=99
> --name racnode2
> oracle/database-rac:21.3.0

0dfa77c7f8f8b5e3bc1d5b2b2739448743dcb98a0e660ede087281b6a6aa0324

[root@OEL7 ~]# docker network discoect bridge racnode2
[root@OEL7 ~]# docker network coect rac_pub_nw1 --ip 10.0.20.152 racnode2
[root@OEL7 ~]# docker network coect rac_priv_nw1 --ip 192.168.10.152 racnode2
[root@OEL7 ~]# docker start racnode2
racnode2
[root@OEL7 ~]# docker logs racnode2 -f

Creating env variables file /etc/rac_env_vars
Starting Systemd
systemd 219 ruing in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
Detected virtualization other.
Detected architecture x86-64.
Welcome to Oracle Linux Server 7.9!
Set hostname to <racnode2>.
[/usr/lib/systemd/system/systemd-pstore.service:22] Unknown lvalue 'StateDirectory' in section 'Service'
Caot add dependency job for unit display-manager.service, ignoring: Unit not found.
[ OK ] Reached target Local Encrypted Volumes.
[ OK ] Created slice Root Slice.
[ OK ] Listening on /dev/initctl Compatibility Named Pipe.
[ OK ] Started Dispatch Password Requests to Console Directory Watch.
[ OK ] Reached target RPC Port Mapper.
[ OK ] Listening on Delayed Shutdown Socket.
[ OK ] Listening on Joual Socket.
[ OK ] Created slice User and Session Slice.
[ OK ] Started Forward Password Requests to Wall Directory Watch.
[ OK ] Created slice System Slice. Starting Joual Service... Starting Read and set NIS domaiame from /etc/sysconfig/network... Starting Configure read-only root support... Starting Rebuild Hardware Database...
[ OK ] Reached target Slices.
[ OK ] Reached target Local File Systems (Pre).
[ OK ] Created slice system-getty.slice.
[ OK ] Reached target Swap.
[ OK ] Started Joual Service.
[ OK ] Started Read and set NIS domaiame from /etc/sysconfig/network. Starting Flush Joual to Persistent Storage...
[ OK ] Started Configure read-only root support.
[ OK ] Reached target Local File Systems. Starting Mark the need to relabel after reboot... Starting Preprocess NFS configuration... Starting Rebuild Joual Catalog... Starting Load/Save Random Seed...
[ OK ] Started Load/Save Random Seed.
[ OK ] Started Mark the need to relabel after reboot.
[ OK ] Started Preprocess NFS configuration.
[ OK ] Started Flush Joual to Persistent Storage. Starting Create Volatile Files and Directories...
[ OK ] Started Rebuild Joual Catalog.
[ OK ] Started Create Volatile Files and Directories. Mounting RPC Pipe File System... Starting Update UTMP about System Boot/Shutdown...
[FAILED] Failed to mount RPC Pipe File System.
See 'systemctl status var-lib-nfs-rpc_pipefs.mount' for details.
[DEPEND] Dependency failed for rpc_pipefs.target.
[DEPEND] Dependency failed for RPC security service for NFS client and server.
[ OK ] Started Update UTMP about System Boot/Shutdown.
[ OK ] Started Rebuild Hardware Database. Starting Update is Completed...
[ OK ] Started Update is Completed.
[ OK ] Reached target System Initialization.
[ OK ] Listening on D-Bus System Message Bus Socket.
[ OK ] Started Daily Cleanup of Temporary Directories.
[ OK ] Reached target Timers.
[ OK ] Listening on RPCbind Server Activation Socket. Starting RPC bind service...
[ OK ] Reached target Sockets.
[ OK ] Started Flexible branding.
[ OK ] Reached target Paths.
[ OK ] Reached target Basic System. Starting LSB: Bring up/down networking...
[ OK ] Started D-Bus System Message Bus. Starting Self Monitoring and Reporting Technology (SMART) Daemon... Starting Login Service... Starting Resets System Activity Logs... Starting GSSAPI Proxy Daemon... Starting OpenSSH Server Key Generation...
[ OK ] Started RPC bind service. Starting Cleanup of Temporary Directories...
[ OK ] Started Cleanup of Temporary Directories.
[ OK ] Started Login Service.
[ OK ] Started Resets System Activity Logs.
[ OK ] Started GSSAPI Proxy Daemon.
[ OK ] Reached target NFS client services.
[ OK ] Reached target Remote File Systems (Pre).
[ OK ] Reached target Remote File Systems. Starting Permit User Sessions...
[ OK ] Started Permit User Sessions.
[ OK ] Started Command Scheduler.
[ OK ] Started OpenSSH Server Key Generation.
[ OK ] Started LSB: Bring up/down networking.
[ OK ] Reached target Network. Starting OpenSSH server daemon... Starting /etc/rc.d/rc.local Compatibility...
[ OK ] Reached target Network is Online. Starting Notify NFS peers of a restart...
[ OK ] Started Notify NFS peers of a restart.
[ OK ] Started OpenSSH server daemon.
[ OK ] Started /etc/rc.d/rc.local Compatibility.
[ OK ] Started Console Getty.
[ OK ] Reached target Login Prompts.
10-05-2022 15:29:44 +0330 : : Process id of the program :
10-05-2022 15:29:44 +0330 : : #################################################
10-05-2022 15:29:44 +0330 : : Starting Grid Installation
10-05-2022 15:29:44 +0330 : : #################################################
10-05-2022 15:29:44 +0330 : : Pre-Grid Setup steps are in process
10-05-2022 15:29:44 +0330 : : Process id of the program :
10-05-2022 15:29:44 +0330 : : Disable failed service var-lib-nfs-rpc_pipefs.mount
10-05-2022 15:29:44 +0330 : : Resetting Failed Services
10-05-2022 15:29:44 +0330 : : Sleeping for 60 seconds
[ OK ] Started Self Monitoring and Reporting Technology (SMART) Daemon.
[ OK ] Reached target Multi-User System.
[ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes...
[ OK ] Started Update UTMP about System Runlevel Changes.
Oracle Linux Server 7.9
Keel 4.14.35-1818.3.3.el7uek.x86_64 on an x86_64
racnode2 login: 10-05-2022 15:30:44 +0330 : : Systemctl state is ruing!
10-05-2022 15:30:44 +0330 : : Setting correct permissions for /bin/ping
10-05-2022 15:30:44 +0330 : : Public IP is set to 10.0.20.152
10-05-2022 15:30:44 +0330 : : RAC Node PUBLIC Hostname is set to racnode2
10-05-2022 15:30:44 +0330 : : Preparing host line for racnode2
10-05-2022 15:30:44 +0330 : : Adding n10.0.20.152tracnode2.example.comtracnode2 to /etc/hosts
10-05-2022 15:30:44 +0330 : : Preparing host line for racnode2-priv
10-05-2022 15:30:44 +0330 : : Adding n192.168.10.152tracnode2-priv.example.comtracnode2-priv to /etc/hosts
10-05-2022 15:30:44 +0330 : : Preparing host line for racnode2-vip
10-05-2022 15:30:44 +0330 : : Adding n10.0.20.162tracnode2-vip.example.comtracnode2-vip to /etc/hosts
10-05-2022 15:30:44 +0330 : : Preparing host line for racnode-scan
10-05-2022 15:30:44 +0330 : : Preapring Dev تالار های تخصصی...
ما را در سایت تالار های تخصصی دنبال می‌کنید
 برچسب:  نویسنده: خنجی بازدید: 147 تاريخ: پنجشنبه 8 دی 1401 ساعت: 12:21