I am building mongodb / mongo on OpenBSD 7.0.
I’ve tried various environment variable tricks (export CC=egcc, etc.), various pathing tricks, but python buildscripts/scons.py install-mongod --disable-warnings-as-errors in my virtual environment won’t find gcc 8.4.0 (/usr/local/bin/egcc) and it won’t find ssl.h (/usr/include/openssl/ssl.h).
Any tips, please?
Hi @Jack_Woehr -
In general, SCons ignores the shell environment in the interest of reproducible builds. If you wish to customize CC and CXX when invoking SCons, you must pass them as command line arguments to SCons. Try the following instead:
python buildscripts/scons.py install-mongod --disable-warnings-as-errors CC=/usr/local/bin/egcc CXX=/usr/local/bin/egc++
(Note: I’m guessing on that CXX value, I’m not very familiar with OpenBSD’s compiler setup).
One more thing I’ll note regarding toolchain is that it looks like your toolchain is in /usr/local/. Will your toolchain automatically arrange for binaries to search for the C++ runtime in the right place? If not, some additional flags may be needed to setup an additional runpath.
The issue OpenSSL is more surprising. Can you provide more details on what is going wrong? Is it failing the configure check? If so, you can look in the config.log, a path to which is printed out after a failed config, and it will often show something helpful. It might be worth building with VERBOSE=1 --config=force to make sure you aren’t just getting echoed back a cached failure.
Also, it would be helpful if you could let me know what branch / version of MongoDB you are trying to build.
Thanks,
Andrew
Thanks, Andrew … trying this now …
Trying to build master f7e3b602cf
Living on the edge!
Please let me know how it works out.
Okay @Andrew_Morrow, that got me past the compiler error (BTW for Gnu it’s CXX=/usr/local/bin/eg++) but stuck with openssl apparently because a dependent library is wanted:
scons: Configure: Checking for SSL_version(NULL) in C library ssl...
build/scons/opt/sconf_temp/conftest_42d562287bca6fadd943ac430ed30054_0.c <-
|
|
|#include "openssl/ssl.h"
|int
|main() {
| SSL_version(NULL);
|return 0;
|}
|
/usr/local/bin/egcc -o build/scons/opt/sconf_temp/conftest_42d562287bca6fadd943ac430ed30054_0.o -c -std=c11 -Werror -fasynchronous-unwind-tables -ggdb -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -fno-omit-frame-pointer -fno-strict-aliasing -O2 -march=sandybridge -mtune=generic -mprefer-vector-width=128 -Wno-unused-local-typedefs -Wno-unused-function -Wno-deprecated-declarations -Wno-unused-const-variable -Wno-unused-but-set-variable -Wno-missing-braces -fstack-protector-strong -fno-builtin-memcmp -fPIE -DNDEBUG build/scons/opt/sconf_temp/conftest_42d562287bca6fadd943ac430ed30054_0.c
/usr/local/bin/egcc -o build/scons/opt/sconf_temp/conftest_42d562287bca6fadd943ac430ed30054_0_a1ba5b757e2b0ce3219a399f52ae31c0 -Wl,--fatal-warnings -pthread -Wl,-z,now -fstack-protector-strong -Wl,--build-id -Wl,--hash-style=gnu -Wl,-z,noexecstack -Wl,--warn-execstack -Wl,-z,relro -Wl,--compress-debug-sections=none -pie -rdynamic build/scons/opt/sconf_temp/conftest_42d562287bca6fadd943ac430ed30054_0.o -lkvm -lcrypto -lssl -lcrypto -ldl
ld: error: unable to find library -ldl
collect2: error: ld returned 1 exit status
scons: Configure: no
There is no libdl on my system … if it’s looking for dlopen() that should be in libc on OpenBSD.
I believe this is happening due to this logic in the build system:
https://github.com/mongodb/mongo/blob/c5837542821a86feeea41e6a00c96ef746f03b15/SConstruct#L3935
MongoDB currently does no testing on any BSDs, so it isn’t surprising to me that there are some incorrect assumptions. Unfortunately, you will need to make a local fix to work past this. It is reasonable to assume that you will find other issues like this. In particular, you may find that several of the vendored third party libraries (MozJS, tcmalloc, and libunwind all come to mind) lack configurations for your platforms. You may need to disable features or build against a system version in order to work past those.
Thanks, Andrew … I didn’t think it was going to be easy 
Well, I made some progress but now it has gone bonkers and is looking for arm_neon.h
scons: Configure: Checking for C header file arm_neon.h...
build/scons/opt/sconf_temp/conftest_ca1da64614a9ba196b2237c2c671e2f3_0.c <-
|
|#include "arm_neon.h"
If I remember right, that is a misleading bit of information that the build spits out. The check for that is unconditional, but failing it doesn’t stop your build. It just happens to be the last check that gets run, so it looks suspicious when you see the “failed” result. I suspect there is another error happening elsewhere.
Yes, @Andrew_Morrow , I’m sure you’re right about that.
It doesn’t seem to make it into the failure log, but watching stdout I see what’s happening.
ISTR speed-reading some MongoDB docs that later versions of the x86_64 port require advanced instructions.
Well.
Here’s some output from the oldish computer on which I have OpenBSD installed that leads me to believe that the immediate problem is that the Intel(R) Core™2 Duo CPU T6500 is unsupported for MongoDB 5:
Compiling build/opt/mongo/db/ttl.o
/tmp//cch4k6rb.s: Assembler messages:
/tmp//cch4k6rb.s:3658: Error: no such instruction: `vzeroupper'
/tmp//cch4k6rb.s:3668: Error: no such instruction: `vzeroupper'
/tmp//cch4k6rb.s:5245: Error: no such instruction: `vzeroupper'
/tmp//cch4k6rb.s:5266: Error: no such instruction: `vzeroupper'
/tmp//cch4k6rb.s:5275: Error: no such instruction: `vzeroupper'
/tmp//cch4k6rb.s:5470: Error: no such instruction: `vzeroupper'
/tmp//cch4k6rb.s:5491: Error: no such instruction: `vzeroupper'
/tmp//cch4k6rb.s:5500: Error: no such instruction: `vzeroupper'
/tmp//cch4k6rb.s:5820: Error: no such instruction: `vmovdqu 16(%rdx),%xmm0'
/tmp//cch4k6rb.s:5821: Error: no such instruction: `vmovaps %xmm0,-96(%rbp)'
/tmp//cch4k6rb.s:6000: Error: no such instruction: `vmovdqa -96(%rbp),%xmm1'
/tmp//cch4k6rb.s:6001: Error: no such instruction: `vmovaps %xmm1,-64(%rbp)'
… and so on for a page or two.
Yeah, that looks like your assembler doesn’t like the codegen from the compiler. However you can override our default targeting of -march=sandybridge by providing your own -march argument to GCC, again on the command line, via CCFLAGS.
scons ... CCFLAGS=-march=core2
That got me farther. Now …
Compiling build/opt/mongo/db/user_write_block_mode_op_observer.o
virtual memory exhausted: Cannot allocate memory
which is funny because I don’t see it hitting swap. It just exhausts free physical RAM.
Building the server sources is fairly resource intensive. I recommend using the most powerful machine you can. But there are a few things you can try to reduce various sorts of resource constraints:
By default the build will use all available cores as found by the python psutil library. But you can reduce that on the command line by passing an explicit -j N argument. Cutting that N value back from using all your local cores should reduce the memory pressure during compilation.
Linking a static mongod binary can take a lot of memory. If you don’t need a production quality binary, you can build with --link-model=dynamic, which will instead build a mongod that links tons of little shared libraries. That makes the final link much less resource intensive.
If you must go with a static build but don’t need debug info after development, you can try building with CCFLAGS=-gsplit-dwarf, which should reduce the amount of memory and disk used to manage debug info, but the resulting build is really only appropriate for development use since you can’t take the debug info with you easily. Note that you are already passing an argument to CCFLAGS, so the syntax for passing multiple values is a space separated string: scons ... CCFLAGS="-march=core2 -gsplit-dwarf". There isn’t much use to -gsplit-dwarf for a --link-model=dynamic build, in our experience.
You can save a disk space on the installation by building with --install-action=hardlink, which will hardlink files into the installation directory, rather than copying them from the build directory. This will only work as long as your build directory and installation directory are on the same filesystem. If you have customized DESTDIR or similar, that may not be true.
All of the above comes with the caveat that all our developer experience with these flags is from macOS and Linux. So, just like with the code itself, your mileage may vary on BSDs.
Andrew
Thanks for all the tips, @Andrew_Morrow … Will work with this today.
You’re right, it’s pretty silly to throw this little old laptop at this big compilation problem.
Of course I’m really using MongoDB on a mixture of more advanced platorms, Ubuntu and Fedora.
In the present case, I’m just trying to see the viability of OpenBSD as a modern MongoDB platform and had installed the latest OBSD cut on a little laptop. Thanks for your patience with my experiment!
PS I couldn’t make it accept --link-model=dynamic so I tried -link-model=dynamic (1 dash) and that is running now and has already gotten further than I got before …
Using CCFLAGS="-march=core2 -g -link-model=dynamic" (-g strips everything 
) I was able to get far enough to run into errors that are based on the code … now I have to do some real work … My system doesn’t like boost::optional<long long> _N; …
@Jack_Woehr - That sounds like you are up and running on a working environment for doing the necessary porting. Please reach out of if you have further questions.
I’d be careful there. We already manage optimization flags like -O2 and we always build with -Wall. The setting -NDEBUG is also something we already manage for you, based on whether the build has the --dbg flag.
I definitely recommend hunting around in the top level SConstruct for these sorts of flags before introducing them via the command line. Yes, the file is huge and complex, but there is a lot of accumulated knowledge baked in there too.
Hi,
Sorry for the slightly OT question, but which debugging options are included within the --dbg flag?