fix CMake build

[Trass3r]

fix #28

[dtzWill]

Awesome, just what I was hoping for, thank you! :)

Unfortunately, I'm running into a problem that seems to come down to a missing file "systemIncludePath.c.tmpl"--is that possible or does that mean I'm doing something else wrong?

[fay59]

Missed the stray LICENSE.txt, thanks for cleaning up.

I'm trying to build on Ubuntu 16.10 with the typical packages installed:

• llvm-3.9
• llvm-3.9-dev
• clang-3.9
• libclang-3.9-dev

and cmake complains that it can't find ClangConfig.cmake/clang-config.cmake (and I can't find any file with that name on that machine either). What are you using? I'd especially like to know for INSTALL instructions.

[Trass3r]

A Classic. I forgot to add the template.
I built the 3.9 branch myself (with assertions enabled).
Looks like the debian package does not include clang's cmake files.

lib/cmake/clang/ClangConfig.cmake
lib/cmake/clang/ClangTargets.cmake

[fay59]

This has been a known issue on Ubuntu's side for a little over 2 years. Do you know how you want to proceed with this?

[fay59]

I mean, what does the statement around building fcd with CMake become? You can't build fcd if you've installed Clang through Ubuntu/Debian's package manager? Should fcd bundle a ClangConfig.cmake that matches the Debian/Ubuntu official distributions?

[Trass3r]

Now linking it manually, expecting the clang libs in the same folder as the llvm ones.
Rather fragile though as the deps between the libs are not handled automatically anymore.

[fay59]

Wait–the newer Ubuntu/Debian packages do include ClangConfig, just in a different location. Sorry about that. (For the record, and since that may come in handy when updating INSTALL, the files are /usr/lib/llvm-3.9/lib/cmake/llvm/LLVMConfig.cmake and /usr/share/llvm-3.9/cmake/ClangConfig.cmake.)

The next issue with the packaged installation is that the actual names of the compilers are clang-3.9 and clang++-3.9. Of course, that throws a wrench in the Makefile any time that it specifically invokes either.

I'm trying to see if the guys who package Clang for Debian an Ubuntu have anything to say on that matter, but if you have an idea, you're free to try it as well. The Makefile currently expects that there will be a suffix to Clang, which is not much better, I guess.

[Trass3r]

Another option (567a25f) is to just build everything with clang (as it is needed to compile some files anyway) and let the user set it via CC=... CXX=... cmake ..

[fay59]

This is basically what the package maintainers said that you should do. It should probably check that the Clang version is 3.9 as well.

Unless you want to sign up for it, right now, no one regularly builds fcd on Linux. I build with AppleClang, which lags behind the latest and greatest that we're now forcing Linux people to use, which quite possibly turn on more warnings/detecting more violations. This makes warnings as errors a moving target on a platform that receives less attention, and that will quickly become a problem. I'm not interested in getting build fail bugs for warnings, or discouraging people from using fcd because a problem that has a 90% chance of being benign was reported by the compiler.

[Trass3r]

Sure. Though a Travis build or something similar would still be a good idea.

[fay59]

I'm looking into that. The biggest reason that I've held off is that fcd's output is not stable across runs (most likely because it uses ASLR'd pointers for keys to a bunch of unordered_maps/unordered_sets), so comparing output can be difficult.

I'm still not a big fan of warnings as errors.

Also, I'd suggest making systemIncludePath.c a C++ file. Otherwise, that's the only C file in the project and it won't build with CMake on a system where GCC is the default C compiler unless CC is specified in addition to CXX.

[fay59]

I've set up Travis. You can merge with master and check if the PR works in a world where people have installed LLVM through their package manager.

[Trass3r]

Looks like it hangs if somethings goes wrong.
omg they still run Ubuntu 12.04, that won't work. Maybe wth libc++.

[fay59]

To be clear, it doesn't hang, it's just the macOS build that takes 45 minutes to prop. (See what you signed me up for? 😉) Luckily for you, the Linux build is much faster, but you need to check individual builds for status, not the whole thing.

They run Ubuntu 12.04 on sudo-less machines and 14.04 on machines that are sudo-enabled. This is the one that we have. The apt.sources that we have give us CMake 3.2, LLVM repositories and recent-enough libstdc++, so you should be good to go.

If you get to "bad decrypt", it means that you got it: that's Travis failing to decrypt the fcd-tests's repository private key because it would be kind of broken if it just let you pretend that you were me.

[fay59]

You could need a trick similar to http://libcxx.llvm.org/docs/UsingLibcxx.html but using libstdc++. For reference, the paths are /usr/include/c++/v6 for includes and /usr/lib/gcc/x86_64-gnu-linux/6 for libraries. (You can also get the package yourself at http://ppa.launchpad.net/ubuntu-toolchain-r/test/ubuntu/pool/main/g/gcc-6/libstdc++-6-dev_6.2.0-3ubuntu11~14.04_amd64.deb and check its structure to get the same information.)

[Trass3r]

I see Ubuntu 12 with GCC 4.6 in the log:
https://travis-ci.org/zneak/fcd/jobs/195962461

[fay59]

You're right, I overlooked that you can get a 12.04 environment even on a sudo build (in fact, I mix Trusty and Precise packages in the .travis.yml file).

[fay59]

The alignof(max_align_t) check is entirely paranoid, so I've removed it. This could unblock the build.

[fay59]

It didn't, so I spun up a 12.04 VM to see what was wrong. Turns out that libstdc++6, which is installed as a requirement for llvm-3.9, does not include libstdc++-6-dev. With that installed, Clang will pick it up automatically. You should add it to the .travis.yml file.

The first next error is:

make[2]: *** No rule to make target `CMakeFiles/emu.dir/x86.emulator.asm.o', needed by `fcd'. Stop.

That's not my area of expertise, but I'm assuming that it's a simple-ish directive error in the CMakeList file.

The second error is:

cc: error: unrecognized option '--system-header-prefix=llvm/'
cc: error: unrecognized option '--system-header-prefix=clang/'
cc: error: unrecognized option '--system-header-prefix=clang-c/'

As I think that I mentioned before, this happens most likely because you made the systemIncludePath file a C file instead of a C++ file, and this causes CMake to select gcc on systems where it is the default compiler instead of Clang. You should make it a C++ file.

[fay59]

Looks like what doesn't work on Travis is my fault now. I'm still getting this error on 12.04 though:

make[2]: *** No rule to make target CMakeFiles/emu.dir/x86.emulator.asm.o', needed by fcd'. Stop.

Looking closer, it's because the cc on that platform doesn't know what to do with asm files:

[ 1%] Building ASM object CMakeFiles/emu.dir/x86.emulator.asm.o
/usr/bin/cc -g -o CMakeFiles/emu.dir/x86.emulator.asm.o -c /tmp/fcd/build/x86.emulator.asm
cc: warning: /tmp/fcd/build/x86.emulator.asm: linker input file unused because linking not done

Changing the extension from .asm to the better-known .s fixed the problem. There is an issue with how I use the ar command to extract .deb archives on Linux in the tests. There is also an issue with the built product, where fcd complains as such:

: CommandLine Error: Option 'help-list' registered more than once!
LLVM ERROR: inconsistency in registered CommandLine options

I'm running out of time and will look at it later.

[fay59]

I don't have solid evidence for it, but my best lead so far is that the output links against both the static and dynamic libraries of LLVM:

$ ldd ./fcd
    linux-vdso.so.1 =>  (0x00007ffefab30000)
    libcapstone.so.3 => /usr/lib/libcapstone.so.3 (0x00007fc719ece000)
    libclang-3.9.so.1 => /usr/lib/x86_64-linux-gnu/libclang-3.9.so.1 (0x00007fc7189c5000)
    libpython2.7.so.1.0 => /usr/lib/libpython2.7.so.1.0 (0x00007fc7184c8000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fc7182c4000)
    libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007fc71809d000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc717e80000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fc717c69000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc71796d000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fc71765a000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fc717443000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc717085000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fc71a389000)
--> libLLVM-3.9.so.1 => /usr/lib/x86_64-linux-gnu/libLLVM-3.9.so.1 (0x00007fc714611000)
    libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007fc7143b2000)
    libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007fc713fd6000)
    libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fc713dd3000)
    libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007fc713bcb000)
    libedit.so.2 => /usr/lib/x86_64-linux-gnu/libedit.so.2 (0x00007fc7139a4000)
    libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007fc713798000)

You're possibly missing a --link-static in the llvm-config invocation.

Also, -lclang should only be used to resolve clang_createTranslationUnit, but it's actually used for a whole lot of symbols (as evidenced by what happens if you put -lclang last in the linker input).

[fay59]

Yes, replacing all LLVM .a libs with -lLLVM-3.9 (which is the result of llvm-config-3.9 --libs ... on that distribution, too) solves the problem. This is an important problem to fix because while fcd built in Travis and died in the test scripts, you'll still notice that every single invocation ended with a status code of 1: the build gives a broken executable.

[Trass3r]

There is no llvm-config invocation. Just the exported cmake config file.
I guess they use the new *DYLIB* mess. http://llvm.org/docs/CMake.html#llvm-specific-variables
Before it was either shared or static. Not sure how this is handled now.

[fay59]

On this installation, the Clang shared library is what pulls in the LLVM shared library. That does not appear to systematically be the case: for instance, the macOS Clang shared library doesn't. I'm successfully getting around the problem with this:

# The Clang cmake config file is broken on binary distributions. We need a way
# to tell if libclang was linked against the LLVM shared library.
execute_process(COMMAND ldd "${LLVM_LIBRARY_DIR}/libclang.so" COMMAND grep -q libLLVM RESULT_VARIABLE linked_to_libllvm)

if (${linked_to_libllvm} EQUAL 0)
	message(STATUS "Link to LLVM shared library.")
	set(llvm_libs -lLLVM-3.9)
else()
	message(STATUS "Link to LLVM static libraries.")
	llvm_map_components_to_libnames(llvm_libs analysis asmparser bitreader codegen core coverage instcombine instrumentation ipo irreader linker mc mcparser object option passes profiledata scalaropts support target transformutils vectorize)
endif()

# Ubuntu does not package ClangConfig
target_link_libraries(fcd "-L${LLVM_LIBRARY_DIR}" clangIndex clangCodeGen clangFormat clangToolingCore clangRewrite clangFrontend clangDriver clangParse clangSerialization clangSema clangEdit clangAnalysis clangAST clangLex clangBasic)
target_link_libraries(fcd ${llvm_libs} capstone clang -Wl,--gc-sections)

This also adds a small number of static libraries to the list, because the static Clang libraries need symbols that were otherwise pulled from the LLVM shared library that the Clang shared library brought in. No idea how that builds on systems where it doesn't do that.

The result executable is broken in a different way (it systematically crashes inside of ASTUnit::LoadFromCompilerInvocation while destroying a string). I guess that's progress.

[Trass3r]

Why would that make a difference if libclang is linked against libllvm? We don't even call any functions from it.
I guess it would be best to get rid of the whole libclang hack if possible. The CMake script is already complicated enough, and yet unportable.

[fay59]

I don't know why the state of libLLVM leaks into the state of fcd when fcd links against static libraries. It seems to me that they shouldn't interact, unless the linker always prefers symbols from shared libraries when it can find them in both static and shared libraries.

The reason the libLLVM code gets executed even though we never execute anything from libclang, though, is that command-line option registration, among many other things with LLVM, happens in the constructor of global variables. These are always executed regardless of what else you'd do with the library.

I'm not opposed to finding a different solution to get Clang resources. However, I can't think of one.

[fay59]

In the meantime, I get a working executable out of the CMake build on Ubuntu 16.04, so the issues appear to be specific to how we try to build on Trusty. My best bet is that the LLVM binaries were built with a different libstdc++ that has an incompatible string layout.

In fact, on Trusty, the LLVM apt packages are happy depending on just libstdc++ 4.8. If that's the case on Precise too, then trying to link fcd against libstdc++ 6 when LLVM links against 4.8 is sure to cause problems.

The other thing is that libstdc++ 4.8 is not C++14-ready. It doesn't have std::align (although we don't strictly need it), and it has weird problems where it doesn't know that it can't pull gets from stdio.h. This can probably be worked around, but it's not particularly exciting that we need to jump through so many hoops to get fcd to work on Travis.

[fay59]

The gets problem is a bug in libstdc++ and will be worked around in the nastiest way by Clang 4 when it officially comes out in a few weeks. If you think that the changes that I have on top of yours in the cmake branch are alright, I'm fine with merging it and waiting until then to fix the Linux build on Travis.

[Trass3r]

Looks like it's possible to use 16.04 via Docket rust-lang/rust#34016
I don't have any experience with that though.
Would be nice if we didn't have to work around problems of an outdated system.

[fay59]

You know that I agree. However, I spent about as much time as I was willing to spend on this at the moment.

[Trass3r]

Yeah me too. I guess let's go with this :)

[fay59]

I've manually merged your master branch into this repo's cmake branch already, so I'm closing this PR without automatic merging.