Add CI for Solaris / Illumos

[gnzlbg]

We probably should add CI for Illumos / Sun Solaris, or both. I have no idea about any of these two OSes, but I can help any interested party.

If there is no CI system like Travis, Azure Pipelines, Appveyor, or Cirrus-CI that provides native Solaris and/or Illumos VMs, the best way forward is to test Solaris and Illumos under Linux.

For testing other similar targets under Linux we cross-compile from Linux to the target (AFAIK this is possible for both Solaris and Illumos), and then use qemu or kvm to load a boot image, and run the tests there. See for example the s390x/sparc64 ci scripts, which use qemu-system (the old freebsd scripts used this as well), or this redox PR (#1403) which uses the redoxer crate, which internally uses kvm.

cc ing users that have contributed to these targets: @robertdfrench @bgermann @nbaksalyar @jasonbking @papertigers @vojtechkral @binarycrusader

[robertdfrench]

@gnzlbg there are no commercial CI services for this, but I would be willing to make a host available. I have some experience developing GitHub webhook integrations. These OSes have a construct called "Zones" (not unlike Linux Containers or BSD Jails), and it should not be overly difficult to rig up a scheme where each push to a PR is executed in unique zone. This would be similar in spirit to the Travis CI workflow.

The obvious downside is that the reliability of such a thing would be best-effort on my part, and you may not want to risk me being able to gum-up unrelated PRs.

[papertigers]

First off, thanks for opening this issue!
Secondly, I am speaking from the illumos side of things.

Just so I understand correctly, is there local infrastructure for running a KVM image available, or are we dependent on one of the providers you listed?

If we can provide an illumos image for tests, that would be the easiest path forward. The last time I tried to cross compile from linux, it results in linker errors. The cross compile was attempting to use illumos ld linker flags with the gnu linker.

Also worth noting is rust-lang/rust#55553 -- which is still being worked on but we haven't had much time to dedicate to that effort recently. @jasonbking would know more on that front.

If it's a matter of providing infrastructure, Joyent could probably host some sort of build environment.

Personally I think it would be nicer if we could get one of the above mentioned ci/cd companies to support an official illumos VM. Then others could take advantage of running tests for their projects somewhere. I would be interested in helping on that front if any of those companies want to purse that. We have been working on an Azure OmniOS (aka distro of illumos) that would be a great first step for Azure Pipelines.

[gnzlbg]

@robertdfrench that certainly is an option!

The obvious downside is that the reliability of such a thing would be best-effort on my part, and you may not want to risk me being able to gum-up unrelated PRs.

Yes, I think we prefer something that can run on one or more providers. If it could run on Linux, inside Docker, that would be great for allowing other Rust developers to debug issues on this platforms quicker.

---

@papertigers

is there local infrastructure for running a KVM image available, or are we dependent on one of the providers you listed?

Dependent on one of the providers. Using KVM and qemu is possible on Travis-CI. I assumed that this was possible on Azure Pipelines, but I don't know whether this is the case.

The cross compile was attempting to use illumos ld linker flags with the gnu linker.

Were those errors due to rustc, or due to the C cross-compilation toolchain ? If there is no C cross-compilation toolchain, then cross-compiling isn't probably a good option.

Personally I think it would be nicer if we could get one of the above mentioned ci/cd companies to support an official illumos VM.

That would be great. Beyond Azure Pipelines, Cirrus-CI can use VMs available in Google's Compute Engine as well.

[bgermann]

Joyent's CTO Bryan Cantrill fell in love with Rust, so there should be a chance of getting a sponsored VM or zone from them.

[robertdfrench]

Just to follow up on this, I checked with Cirrus CI, and they said they would be happy to provide OmniOS VMs for open source if someone can produce a GCP image. I pinged the OmniOS mailing list to see if anyone had thoughts on that, and heard from a Dan M about a set of instructions for OmniOS on Azure that could potentially be dusted off and revived (is this what you were referencing @papertigers ?).

Another route would be to use the existing OmniOS image for AWS (https://omniosce.org/setup/aws.html) and see if we can get Cirrus to give us an illumos build of their agent. This would not be available as part of the Cirrus "free tier", but I would be happy to set up and sponsor a restricted AWS account where these can run until we work out a more formal arrangement.

[jasonbking]

Currently, I'm aware of two issues that are blocking rust-lang/rust#55553 (there might be more, but I cannot get any further until these are resolved):

The first one is that libtest has duplicated the functionality of the num_cpus crate. While the num_cpus crate supports the illumos target, libtest's num_cpus does not, so it fails to build with the illumos target enabled.

The other (and larger) issue is that the bundled LLVM w/ the current beta and master rust branches fails to build on illumos (largely around linker issues -- unfortunately since LLVM isn't officially supported on illumos, this isn't that uncommon for newer releases to require some work).

[binarycrusader]

I'm no longer involved in Solaris, but @alanc might know who is for this.

[papertigers]

@robertdfrench yeah I am referring to the work by @danmcd and others in the OmniOS community. If someone has a contact at Azure, or more specifically someone at Azure working on Azure pipelines it would be great to sync up with them and figure out what we would need to do, to become an official supported option.

There's also some on going work to get illumos working properly on GCE which is being driven by the desire to provide illumos golang build resources. So this is always an option as well if we want to use Cirrus CI at some point.

Also just to note most of us replying to this thread are from the illumos community. Not sure if there is much interest from Solaris itself, or if they have resources to help the rust on Solaris effort. I mention this because the two have diverged and will only continue to do so. This is one of the reasons @jasonbking is trying to drive forward a proper illumos target for rustlang itself. The libc crate has already been updated to mention illumos and solaris targets ahead of time.

[alanc]

@psumbera is maintaining the rust packages we ship in Solaris, which you can see at https://github.com/oracle/solaris-userland/tree/master/components/rust

Our primary interest is in being able to keep building Mozilla & GNOME desktop software for Solaris, as they grow more and more rust dependencies.

There is support for running Solaris x86 in the Oracle Cloud, for whatever that's worth.

[gnzlbg]

There is support for running Solaris x86 in the Oracle Cloud, for whatever that's worth.

So that's interesting. It seems that one can try that for free. Maybe it would be possible to use it for CI. I briefly looked if any solaris project is doing that via the github API, but that does not appear to be the case. I've never used the github API, so I don't know how hard would it be to set that up.

[jasonbking]

Another issue is several components are explicitly requiring older versions of rand that are broken on illumos (due to rust-random/rand#637 -- fixed in Feb):

libstd requires 0.6.1
test-cargo-miri requires 0.6.5
rustc-workspace-hack requires 0.6.1

I don't know the reasoning on using those versions (instead of relying on semver). Until those are resolved, we have to go into .cargo/.... and manually patch the old rand source (otherwise things die with an illegal system call error).

[gnzlbg]

For building libc one only needs libcore. If libc doesn't build properly (or isn't properly tested), getting libstd working is often a bit harder - getting libc to work often requires making sure that some system libraries are properly linked.

[jasonbking]

But you cannot get a working rustc on illumos at all without fixing the above issues..