eMedia Hosting

Hi Sumit, when can we meet and discuss making this update?

Sumit, I have a small project for your Linux guys to help us with: I understand there is a 30% speed boost from using http/2 in NGNX. I noticed we do not have that enabled in the version we use. Can we upgrade NGINX and do some performance comparison? https://serverfault.com/questions/765258/use-http-2-0-between-nginx-reverse-proxy-and-backend-webserver. Reach out to me for the SSL key login

I'm creating the Backup entries for clients in the other Backup server (m47)

Bad news from the servers. m46 that it is hosting one of the Backup machines (bk-ny) it is in state of SUSPENDED on Zpool. 2 drives marked as Faulted and 1 as Degradated. Mount is not usable. Tried the zpool tool to recover but after almost 1 hr it failed: Message from syslogd@localhost at Aug 30 15:23:49 ... kernel:PANIC at dmu.c:1017:dmu_write()

Hi, can I set up a meeting this friday?

Sure, let's please scheudle meeting on Monday.

Sumit, can we setup a meeting to get to know your Linux admin?

Cristobal, Migrate users -> eMediaFinder.com to eMediaLibrary.com

Cristobal, please re-enable a way to deploy and monitor sites from old eMediaFinder.com docker instance. rename to sites.emedialibrary.com

Cristobal, can you fix this to redirect to eMediaLibrary: http://www.entermediasoftware.com/

Cristobal, can you provide Sumit access to our server. Sumit please do not share this info with anyone without me approval

Hi Chris, Can you please shre the server details where we need to perform above actions?

Need to shut down the Indian server hosted on Digital Ocean Need to install Ubuntu on one of the spare servers Need to run regular linux updates on all the servers Need to document how to setup zfs/xfs and JBOD drives from scratch Need to check on ansible setup Need to check that all servers have root check installed Need to check on all SMART drives status

will take a bit to migrate it but will get rid of those containers

I'm launching a different server for all TestU stuff

ok

Ian we are shutting down a runaway instance of m38 testu77

951 enterme+ 20 0 70732 57900 4552 R 99.7 0.1 0:44.59 pdftoppm -f 0 -l 0 -aa no -aaVector no -rx 300 -ry 300 -png /opt/entermediadb/webapp/WEB-INF/data/testu/system/originals/Legacy/1100/36___121.pdf poppler 2360 root 20 0 112680 57432 57104 S 0.3 0.1 0:12.72 /usr/lib/systemd/systemd-journald 34222 enterme+ 20 0 68252 55132 4484 R 100.0 0.1 0:35.90 pdftoppm -f 0 -l 0 -aa no -aaVector no -rx 300 -ry 300 -png /opt/entermediadb/webapp/WEB-INF/data/testu/system/originals/Legacy/1158/76___121.pdf poppler 6625 root 20 0 2673664 54248 15384 S 0.0 0.1 1:24.72 /usr/bin/containerd 1951 enterme+ 20 0 57344 43940 4816 R 100.0 0.0 43:41.15 pdftoppm -f 0 -l 0 -aa no -aaVector no -rx 300 -ry 300 -png /opt/entermediadb/webapp/WEB-INF/data/testu/system/originals/Legacy/38/FCOM 19 Mar 2018 Complete 2.pdf poppler 6603 root 20 0 289948 35028 31452 S 0.0 0.0 0:08.38 /usr/sbin/rsyslogd

And try to get a rsync while it's up just in case

I will prune some stuff I don't really need anyway, keep it to bare min

ok

Its likely a power supply issue. I will have them try and find the issue. But likely this server is dead for now

I will wait a few min to start docker? See if it crashes independant of that?

I'm on agian

I wonder if there are any crash logs or anything. Docker shouldn't take down the entire server

ok

It shows a screen shot of the server booting now

CAn you login to ssh via that console or anything?

certainly seems different than anything I've seen before

I will have them look deeper

We have a console login for it and it says its on: Host is currently on

I remounted ZFS, started docker and everything looked fine
then it stopped again

I really don't know

Some kind of hardware problem?

And now I can't get in again

ok done, it should recover now

I can fix the mount one sec

I see this in the boot log: [ OK ] Started Load Kernel Modules. Starting Apply Kernel Variables... Starting Configure read-only root support... Starting Rebuild Hardware Database... Activating swap /swapfile... Starting Flush Journal to Persistent Storage... Starting Mount ZFS filesystems... Starting udev Kernel Device Manager... [ OK ] Started Apply Kernel Variables. [ OK ] Started Configure read-only root support. [FAILED] Failed to activate swap /swapfile. See 'systemctl status swapfile.swap' for details. [DEPEND] Dependency failed for Swap. Starting Load/Save Random Seed... [FAILED] Failed to start Mount ZFS filesystems. See 'systemctl status zfs-mount.service' for details. [ OK ] Started Load/Save Random Seed.

yeah it created a ton of empty dockers

I think I can fix it, there is a script here

Soemthign is still wrong, I think the drive didn't mount
ZFS

it's been running fine

I hope we dont have a hardware issue

no just certificates and upgrades

So nothin weird there

root: cd 990 cd 991 cat ~/.bash_history 992 exit 993 exit 994 history 995 exit 996 ip addr 997 ip addr | less 998 ping 8.8.8.8 999 exit

Last history logs look like this: wget -O http://dev.ijsolutions.ca/jenkins/jo/colony42/lastSuccessfulBuild/artifact/deploy/u[A 990 nano olw.sh 991 ./olw.sh 992 cat upgrade.xml 993 clear 994 ./logs.sh 995 ./olw.sh 996 cat upgrade.xml 997 ./olw.sh 998 sudo cerbot --nginx -d oneliveweb.com 999 sudo certbot --redirect --nginx -d oneliveweb.com 1000 sudo certbot --nginx -d beta.testu.co

must have, I just noticed and couldn't ssh in

Perhaps it crashed

They said it was a software issue. Why did it get shut down?

I can get rid of a few but I don't think that was the cause

I got rid of a slew of them the other day - That won't be the problem, it's been running like this for at least 2-3 years

Can we remove some of these docker containers?
M38

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 32298cc8bf90 entermediadb/entermedia10:latest "/usr/bin/entermedia…" 6 days ago Up 8 minutes colony4242 40977a5d4f68 1b73f2d79127 "/usr/bin/entermedia…" 3 weeks ago Up 8 minutes cheese67 dd40f179557e 1b73f2d79127 "/usr/bin/entermedia…" 3 weeks ago Up 8 minutes 0.0.0.0:1990->1935/tcp, :::1990->1935/tcp paes.testu.co90 65d88d160902 1b73f2d79127 "/usr/bin/entermedia…" 3 weeks ago Up 8 minutes 0.0.0.0:1983->1935/tcp, :::1983->1935/tcp demotestu83 4b47efae42be 1b73f2d79127 "/usr/bin/entermedia…" 3 weeks ago Up 8 minutes 0.0.0.0:1978->1935/tcp, :::1978->1935/tcp testu-dev78 60dae95aab47 1b73f2d79127 "/usr/bin/entermedia…" 3 weeks ago Up 8 minutes 0.0.0.0:1960->1935/tcp, :::1960->1935/tcp oneliveweb60 1c0aac6d54e1 jenkins/jenkins:lts "/usr/bin/tini -- /u…" 4 weeks ago Restarting (1) 28 seconds ago oneliveweb25 e2d6bb2ab055 990f0df788a1 "/usr/bin/entermedia…" 8 months ago Up 8 minutes willow65 851fb91829a2 1a958a4539d4 "/usr/bin/entermedia…" 8 months ago Up 8 minutes evolvedlearning10 11406e94554d entermediadb/entermediadb9:latest "/usr/bin/entermedia…" 12 months ago Up 8 minutes connect53 3cba3983389d c747f15bf8e7 "/usr/bin/entermedia…" 17 months ago Up 8 minutes netevolved55 a752d3d00110 993aa438e36b "/usr/bin/entermedia…" 17 months ago Up 8 minutes testu77 8b46e0a3901c 29d4fb0c3848 "/usr/bin/entermedia…" 18 months ago Up 8 minutes testreadypro11 02e054a1c427 ed7fa90d9dde "/usr/bin/entermedia…" 3 years ago Up 8 minutes agbu99 23b0ac80785f gcr.io/google-containers/cadvisor "/usr/bin/cadvisor -…" 3 years ago Up 8 minutes (healthy) 0.0.0.0:9110->8080/tcp, :::9110->8080/tcp cadvisor ba7c96a921fb 10f2ac0d4464 "/usr/bin/entermedia…" 3 years ago Up 8 minutes metroeastb181 50e77dfdd5a7 10f2ac0d4464 "/usr/bin/entermedia…" 3 years ago Up 8 minutes almb179 195c0fd9272b f53a12974117 "/usr/bin/entermedia…" 4 years ago Up 8 minutes dpci285 ad52fa89164e da3396d818b7 "/usr/bin/entermedia…" 4 years ago Up 8 minutes alm_dev102 78454a245732 da3396d818b7 "/usr/bin/entermedia…" 4 years ago Up 8 minutes metroeast100