Deprecated: Return type of WP_Theme::offsetExists($offset) should either be compatible with ArrayAccess::offsetExists(mixed $offset): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/class-wp-theme.php on line 554

Deprecated: Return type of WP_Theme::offsetGet($offset) should either be compatible with ArrayAccess::offsetGet(mixed $offset): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/class-wp-theme.php on line 595

Deprecated: Return type of WP_Theme::offsetSet($offset, $value) should either be compatible with ArrayAccess::offsetSet(mixed $offset, mixed $value): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/class-wp-theme.php on line 535

Deprecated: Return type of WP_Theme::offsetUnset($offset) should either be compatible with ArrayAccess::offsetUnset(mixed $offset): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/class-wp-theme.php on line 544

Deprecated: Return type of WP_REST_Request::offsetExists($offset) should either be compatible with ArrayAccess::offsetExists(mixed $offset): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/rest-api/class-wp-rest-request.php on line 960

Deprecated: Return type of WP_REST_Request::offsetGet($offset) should either be compatible with ArrayAccess::offsetGet(mixed $offset): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/rest-api/class-wp-rest-request.php on line 980

Deprecated: Return type of WP_REST_Request::offsetSet($offset, $value) should either be compatible with ArrayAccess::offsetSet(mixed $offset, mixed $value): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/rest-api/class-wp-rest-request.php on line 992

Deprecated: Return type of WP_REST_Request::offsetUnset($offset) should either be compatible with ArrayAccess::offsetUnset(mixed $offset): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/rest-api/class-wp-rest-request.php on line 1003

Deprecated: Return type of WP_Block_List::current() should either be compatible with Iterator::current(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/class-wp-block-list.php on line 151

Deprecated: Return type of WP_Block_List::next() should either be compatible with Iterator::next(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/class-wp-block-list.php on line 175

Deprecated: Return type of WP_Block_List::key() should either be compatible with Iterator::key(): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/class-wp-block-list.php on line 164

Deprecated: Return type of WP_Block_List::valid() should either be compatible with Iterator::valid(): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/class-wp-block-list.php on line 186

Deprecated: Return type of WP_Block_List::rewind() should either be compatible with Iterator::rewind(): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/class-wp-block-list.php on line 138

Deprecated: Return type of WP_Block_List::offsetExists($index) should either be compatible with ArrayAccess::offsetExists(mixed $offset): bool, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/class-wp-block-list.php on line 75

Deprecated: Return type of WP_Block_List::offsetGet($index) should either be compatible with ArrayAccess::offsetGet(mixed $offset): mixed, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/class-wp-block-list.php on line 89

Deprecated: Return type of WP_Block_List::offsetSet($index, $value) should either be compatible with ArrayAccess::offsetSet(mixed $offset, mixed $value): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/class-wp-block-list.php on line 110

Deprecated: Return type of WP_Block_List::offsetUnset($index) should either be compatible with ArrayAccess::offsetUnset(mixed $offset): void, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/class-wp-block-list.php on line 127

Deprecated: Return type of WP_Block_List::count() should either be compatible with Countable::count(): int, or the #[\ReturnTypeWillChange] attribute should be used to temporarily suppress the notice in /www/johnmark_507/public/wp-includes/class-wp-block-list.php on line 199

Deprecated: Constant FILTER_SANITIZE_STRING is deprecated in /www/johnmark_507/public/wp-content/plugins/wordpress-seo/src/conditionals/third-party/elementor-edit-conditional.php on line 22

Deprecated: Constant FILTER_SANITIZE_STRING is deprecated in /www/johnmark_507/public/wp-content/plugins/wordpress-seo/src/conditionals/third-party/elementor-edit-conditional.php on line 28

Deprecated: trim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-includes/class-wp.php on line 173

Deprecated: ltrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-includes/wp-db.php on line 3030
Kubernetes - Cluster Setup Guide - John Mark C
Deprecated: parse_str(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-includes/formatting.php on line 4975

Deprecated: strtolower(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/plugins/wordpress-seo/src/generators/schema-generator.php on line 176

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

John Mark Causing

System Administrator | Hosting Support Engineer

johnmarkcausing@gmail.com

Bacolod City, Philippines

+639393497018

John Mark Causing

System Administrator | Hosting Support Engineer

johnmarkcausing@gmail.com

Bacolod City, Philippines

+639393497018

Overview:

Setting up the servers

I am using multipass in Windows to create linux (Ubuntu) virtual machines.

Note: Make sure that master node k8s-master has more than 1 CPU. Otherwise, this setup will not work (kubeadm init)

Install and setup containerd

Requirements:

Enable kernel modules when the server starts up: overlay and br_netfilter

cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

Make the changes take effect immediately without having to restart the servers.

sudo modprobe overlay
sudo modprobe br_netfilter

Setup system-level settings for kubernetes config: (required in order for the container networking to work!)

cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

Make the changes take effect immediately

sudo sysctl --system

Install the containerd package

sudo apt-get update && sudo apt-get install -y containerd

Create a directory for containerd configuration file and

sudo mkdir -p /etc/containerd

Use the containerd config default command to generate a default configuration and pass that configuration inside the directory you just created.

sudo containerd config default | sudo tee /etc/containerd/config.toml

Restart and check status of containerd

sudo systemctl restart contaierd; systemctl status containerd

Install Setup Kubernetes packages

Disable swap first!

Kubernetes requires to disable swap and make sure the changes persist after a system restart. Run the command:

sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

The second command it will just comment out “#” any line in fstab that has the ‘swap’ word to make sure that swap is removed forever!

Install required packages (apt-transport-https and curl) for the setup process

sudo apt-get update && sudo apt-get install -y apt-transport-https curl

Setup Kubernetes package repository

Add GPG key for Kubernetes package

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

Setup Kubernetest package repository. Add it to the repo!

cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF

Update the package listings

sudo apt-get update

Finally! Let’s install the Kubernetes packages.

Install: kubelet kubadm and kubectl

sudo apt-get install -y kubelet kubeadm kubectl

Initialize the Cluster!

Note: Do this only on the master server (control plane server)

Initialize the Kubernetes cluster on the control plane node using kubeadm.

kubeadm init --pod-network-cidr 192.168.0.0/16

The -pod-network-cidr 192.168.0.0/16 is the network configuration that our Kubernetes network plugin (Calico) is going to need later on

Setup kube config so that you can interact with the cluster using kubectl – You will see the commands after you ran the kubeadm init command above.

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Check if you can get the cluster info using kubectl

kubectl get nodes
Kubectl is able to successfully communicate with the cluster (NotReady means we did not setup yet the networking plugin)

Install the network add-on

Install the Calico network add-on in your cluster.

kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml    
kubectl get nodes

Screenshot below shows that the control plane master is now on the ready status!

Join the worker servers/nodes

It is now time to join the two worker node servers to the cluster. Get the join command from the control plane. You can get the command from the kubeadm init you ran earlier.

Or simply run this command to get the join command:

kubeadm token create --print-join-command

Run the join command from your worker nodes (node1 and node2) – Run as root!

kubeadm join 172.30.103.118:6443 --token qyhffl.8m6lyr17sehrwudq --discovery-token-ca-cert-hash sha256:fc2218c54b931c01670af80dc8f2b2415f2dcfeda1f9481263277d3366720e88

Check the status from the control plane master node

Use kubectl get nodes on the control plane node to verify that all three nodes are successfully registered and in the READY state.

kubectl get nodes

We’ve successfully created a Kubernetes cluster with a Master node (control plane) and worker nodes (node1 and node2) that joined the master node!

Troubleshooting

Stuck using kubectl get nodes

root@k8s-master:~# kubectl get nodes
Unable to connect to the server: dial tcp 172.30.103.118:6443: i/o timeout

Reset the cluster using kubeadm reset

Then run the kubeadm init command again:

kubeadm init --pod-network-cidr 192.168.0.0/16

Then make sure to copy the .kube config file

kubeadm reset also the other nodes then join!


Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71

Deprecated: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/class-et-builder-element.php on line 19534

Deprecated: strpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /www/johnmark_507/public/wp-content/themes/Divi/includes/builder/module/helpers/HoverOptions.php on line 71