Issues upgrading to Mongodb v7 on Ubuntu 20 - Failed with result 'core-dump'

Ops and Admin Installation & Upgrades
,
1

I was upgrading from v4 to v7 and now can’t get past ‘core-dump’

I started by removing anything mongodb related from the server

user@monitor:~$ sudo apt remove --purge mongodb-org-*
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'mongodb-org-unstable-tools' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-mongos' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-tools' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-unstable-database-tools-extra' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-unstable-server' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-unstable-shell' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-shell' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-unstable-mongos' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-database-tools-extra' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-unstable' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-tools-unstable' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-database' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-server' for glob 'mongodb-org-*'
Package 'mongodb-org-unstable' is not installed, so not removed
Package 'mongodb-org-unstable-database-tools-extra' is not installed, so not removed
Package 'mongodb-org-unstable-mongos' is not installed, so not removed
Package 'mongodb-org-unstable-server' is not installed, so not removed
Package 'mongodb-org-unstable-shell' is not installed, so not removed
Package 'mongodb-org-unstable-tools' is not installed, so not removed
Package 'mongodb-org-tools-unstable' is not installed, so not removed
Package 'mongodb-org-database-tools-extra' is not installed, so not removed
Package 'mongodb-org-database' is not installed, so not removed
Package 'mongodb-org-shell' is not installed, so not removed
Package 'mongodb-org-tools' is not installed, so not removed
The following packages will be REMOVED:
  mongodb-org-mongos* mongodb-org-server*
0 upgraded, 0 newly installed, 2 to remove and 0 not upgraded.
After this operation, 311 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 141132 files and directories currently installed.)
Removing mongodb-org-mongos (7.0.2) ...
Removing mongodb-org-server (7.0.2) ...
Processing triggers for man-db (2.9.1-1) ...
(Reading database ... 141115 files and directories currently installed.)
Purging configuration files for mongodb-org-server (7.0.2) ...

Then I check that there’s nothing left installed mongo-related

user@monitor:~$ sudo apt list --installed | grep mongo

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

I added the new apt list entry:

user@monitor:~$ cat /etc/apt/sources.list.d/mongodb-org-7.0.list
deb [ arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-7.0.gpg ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/7.0 multiverse

I was upgrading from v4 to v7 and now can’t get past ‘core-dump’

I started by removing anything mongodb related from the server

user@monitor:~$ sudo apt remove --purge mongodb-org-*
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'mongodb-org-unstable-tools' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-mongos' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-tools' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-unstable-database-tools-extra' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-unstable-server' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-unstable-shell' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-shell' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-unstable-mongos' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-database-tools-extra' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-unstable' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-tools-unstable' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-database' for glob 'mongodb-org-*'
Note, selecting 'mongodb-org-server' for glob 'mongodb-org-*'
Package 'mongodb-org-unstable' is not installed, so not removed
Package 'mongodb-org-unstable-database-tools-extra' is not installed, so not removed
Package 'mongodb-org-unstable-mongos' is not installed, so not removed
Package 'mongodb-org-unstable-server' is not installed, so not removed
Package 'mongodb-org-unstable-shell' is not installed, so not removed
Package 'mongodb-org-unstable-tools' is not installed, so not removed
Package 'mongodb-org-tools-unstable' is not installed, so not removed
Package 'mongodb-org-database-tools-extra' is not installed, so not removed
Package 'mongodb-org-database' is not installed, so not removed
Package 'mongodb-org-shell' is not installed, so not removed
Package 'mongodb-org-tools' is not installed, so not removed
The following packages will be REMOVED:
  mongodb-org-mongos* mongodb-org-server*
0 upgraded, 0 newly installed, 2 to remove and 0 not upgraded.
After this operation, 311 MB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 141132 files and directories currently installed.)
Removing mongodb-org-mongos (7.0.2) ...
Removing mongodb-org-server (7.0.2) ...
Processing triggers for man-db (2.9.1-1) ...
(Reading database ... 141115 files and directories currently installed.)
Purging configuration files for mongodb-org-server (7.0.2) ...

Then I check that there’s nothing left installed mongo-related

user@monitor:~$ sudo apt list --installed | grep mongo

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

I added the apt list:

user@monitor:~$ cat /etc/apt/sources.list.d/mongodb-org-7.0.list
deb [ arch=amd64,arm64 signed-by=/usr/share/keyrings/mongodb-server-7.0.gpg ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/7.0 multiverse

and proceeded to install

user@monitor:~$ sudo apt-get install -y mongodb-org
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  mongodb-database-tools mongodb-mongosh mongodb-org-database mongodb-org-database-tools-extra mongodb-org-mongos mongodb-org-server mongodb-org-shell mongodb-org-tools
The following NEW packages will be installed:
  mongodb-database-tools mongodb-mongosh mongodb-org mongodb-org-database mongodb-org-database-tools-extra mongodb-org-mongos mongodb-org-server mongodb-org-shell mongodb-org-tools
0 upgraded, 9 newly installed, 0 to remove and 0 not upgraded.
Need to get 57.5 MB/156 MB of archives.
After this operation, 530 MB of additional disk space will be used.
Get:1 https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/7.0/multiverse amd64 mongodb-org-shell amd64 7.0.2 [3080 B]
Get:2 https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/7.0/multiverse amd64 mongodb-org-server amd64 7.0.2 [34.0 MB]
Get:3 https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/7.0/multiverse amd64 mongodb-org-mongos amd64 7.0.2 [23.6 MB]
Get:4 https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/7.0/multiverse amd64 mongodb-org-database-tools-extra amd64 7.0.2 [7720 B]
Get:5 https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/7.0/multiverse amd64 mongodb-org-database amd64 7.0.2 [3536 B]
Get:6 https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/7.0/multiverse amd64 mongodb-org-tools amd64 7.0.2 [2892 B]
Fetched 57.5 MB in 2s (26.2 MB/s)           
Selecting previously unselected package mongodb-database-tools.
(Reading database ... 141115 files and directories currently installed.)
Preparing to unpack .../0-mongodb-database-tools_100.8.0_amd64.deb ...
Unpacking mongodb-database-tools (100.8.0) ...
Selecting previously unselected package mongodb-mongosh.
Preparing to unpack .../1-mongodb-mongosh_2.0.2_amd64.deb ...
Unpacking mongodb-mongosh (2.0.2) ...
Selecting previously unselected package mongodb-org-shell.
Preparing to unpack .../2-mongodb-org-shell_7.0.2_amd64.deb ...
Unpacking mongodb-org-shell (7.0.2) ...
Selecting previously unselected package mongodb-org-server.
Preparing to unpack .../3-mongodb-org-server_7.0.2_amd64.deb ...
Unpacking mongodb-org-server (7.0.2) ...
Selecting previously unselected package mongodb-org-mongos.
Preparing to unpack .../4-mongodb-org-mongos_7.0.2_amd64.deb ...
Unpacking mongodb-org-mongos (7.0.2) ...
Selecting previously unselected package mongodb-org-database-tools-extra.
Preparing to unpack .../5-mongodb-org-database-tools-extra_7.0.2_amd64.deb ...
Unpacking mongodb-org-database-tools-extra (7.0.2) ...
Selecting previously unselected package mongodb-org-database.
Preparing to unpack .../6-mongodb-org-database_7.0.2_amd64.deb ...
Unpacking mongodb-org-database (7.0.2) ...
Selecting previously unselected package mongodb-org-tools.
Preparing to unpack .../7-mongodb-org-tools_7.0.2_amd64.deb ...
Unpacking mongodb-org-tools (7.0.2) ...
Selecting previously unselected package mongodb-org.
Preparing to unpack .../8-mongodb-org_7.0.2_amd64.deb ...
Unpacking mongodb-org (7.0.2) ...
Setting up mongodb-mongosh (2.0.2) ...
Setting up mongodb-org-server (7.0.2) ...
Setting up mongodb-org-shell (7.0.2) ...
Setting up mongodb-database-tools (100.8.0) ...
Setting up mongodb-org-mongos (7.0.2) ...
Setting up mongodb-org-database-tools-extra (7.0.2) ...
Setting up mongodb-org-database (7.0.2) ...
Setting up mongodb-org-tools (7.0.2) ...
Setting up mongodb-org (7.0.2) ...
Processing triggers for man-db (2.9.1-1) ...

Doesn’t seem like there were any problems, but a fresh install won’t start.

user@monitor:~$ systemctl start mongod
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ===
Authentication is required to start 'mongod.service'.
Authenticating as: user
Password: 
==== AUTHENTICATION COMPLETE ===

user@monitor:~$ systemctl status mongod
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
     Active: failed (Result: core-dump) since Tue 2023-10-17 17:06:38 EDT; 2s ago
       Docs: https://docs.mongodb.org/manual
    Process: 104443 ExecStart=/usr/bin/mongod --config /etc/mongod.conf (code=dumped, signal=ILL)
   Main PID: 104443 (code=dumped, signal=ILL)

Oct 17 17:06:37 monitor systemd[1]: Started MongoDB Database Server.
Oct 17 17:06:38 monitor systemd[1]: mongod.service: Main process exited, code=dumped, status=4/ILL
Oct 17 17:06:38 monitor systemd[1]: mongod.service: Failed with result 'core-dump'.

So I go to look at the logs, but there aren’t any?

user@monitor:~$ sudo ls /var/lib/mongodb

So what gives?

OS info:

user@monitor:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.6 LTS"
2

This has been discussed at length

see Search results for 'signal=ILL' - MongoDB Developer Community Forums

3

Thanks @steevej

I’m attempting to install in a VM on Proxmox, so I’ll provide CPU info for both.

Hypervisor:

root@hypervisor:~# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 60
model name      : Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz

According to intel spec sheet, I have the AVX2 Instruction Set Extension which as far as I can tell means I’m compatible. I also see AVX2 under “flags” when I check /proc/cpuinfo.

When I check CPU info in the VM, I no longer see AVX2 listed under flags.

user@VM:~$ cat /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 6
model name	: Common KVM processor
stepping	: 1
microcode	: 0x1
cpu MHz		: 3392.144
cache size	: 16384 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni cx16 x2apic aes hypervisor lahf_lm cpuid_fault pti

This discussion suggests that changing my CPU type from kvm64 to Host, I can get AVX2 capabilities in the VM. Will report back after some experimentation.

4

From this discussion, it seems like I can safely use “host” cpu type as long as I don’t have high availability set up.

So I went ahead and did that:

image
image1394×428 29.5 KB

Now when I check cpuinfo in the VM, I see AVX2

user@VM:~$ cat /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 60
model name	: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
stepping	: 3
microcode	: 0x27
cpu MHz		: 3392.144
cache size	: 16384 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat umip md_clear arch_capabilities
bugs		: cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs srbds mmio_unknown
bogomips	: 6784.28
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual
power management:

Voila, she works!

user@VM:~$ sudo systemctl start mongod
[sudo] password for user: 
user@VM:~$ sudo systemctl status mongod
● mongod.service - MongoDB Database Server
     Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2023-10-17 21:10:53 EDT; 2min 39s ago
       Docs: https://docs.mongodb.org/manual
   Main PID: 814 (mongod)
     Memory: 187.7M
     CGroup: /system.slice/mongod.service
             └─814 /usr/bin/mongod --config /etc/mongod.conf

Oct 17 21:10:53 monitor systemd[1]: Started MongoDB Database Server.
Oct 17 21:10:54 monitor mongod[814]: {"t":{"$date":"2023-10-18T01:10:54.945Z"},"s":"I",  "c":"CONTROL",  "id":7484500, "ctx":"main","msg":"Environment variable MONGODB_CONFIG_OVERRIDE_NOFORK =>
5

This topic was automatically closed 5 days after the last reply. New replies are no longer allowed.