Question:
a programming language that I am working on, and the VM is written in Rust. Up until Rust nightly 2018-08-17, everything works fine. Starting with the nightly from the 17th, I'm observing various crashes and different program behaviour. For example: On Windows it will either fail with a , or (more on this in a moment). On Linux . Note that the funny segfault output is because the command is started with Ruby, and Ruby installs its own segmentation fault handler. Locally it will usually fail with the same runtime error as observed in Windows above, but sometimes it will segfault. Sometimes it will panic because certain operations are performed using NULL pointers where this is not expected. The last nightly that did not suffer from these problems was Rust 2018-08-16. Stable Rust also works fine. When the segmentation faults happen, they are usually in different places. For example, for one segmentation fault the backtrace is as follows: 0x00007ffff7e12763 in intmalloc () from 0x00007ffff7e13ada in malloc () from 0x0000555555568e6b in alloc::alloc::alloc (layout=...) at <libinko::chunk::Chunk<T::new (capacity=3) at libinko::register::Register::new (amount=3) at libinko::executioncontext::ExecutionContext::fromblock (block=0x7fffdc0971e0, returnregister=Some = {...}) at libinko::vm::machine::Machine::run (self=<optimized out, process=<optimized out) at 0x0000555555568b7f in libinko::vm::machine::Machine::runwitherrorhandling (self=0x55555567d8c0, process=0x7ffff75fcbd0) at 0x00005555555c88a4 in libinko::vm::machine::Machine::startprimarythreads::{{closure}} (process=...) at <libinko::pool::PoolInner<T::process (self=<optimized out, index=0, closure=0x7ffff75fcc60) at 0x00005555555b739d in <libinko::pool::Pool<T::run::{{closure}} () at std::syscommon::backtrace::rustbeginshortbacktrace (f=...) at 0x00005555555cb0dc in std::thread::Builder::spawn::{{closure}}::{{closure}} () at <std::panic::AssertUnwindSafe<Fas core::ops::function::FnOnce<()::callonce (self=..., args=<optimized out) at std::panicking::try::docall (data=<optimized out) at 0x0000555555618a3a in rustmaybecatchpanic () at 0x00005555555ba39b in std::panicking::try (f=...) at std::panic::catchunwind (f=...) at std::thread::Builder::spawn::{{closure}} () at <F as alloc::boxed::FnBox<A::callbox (self=0x55555567db50, args=<optimized out) at 0x00005555556090db in LTdynLTCOutputRu20u20aGTasLTGT::callonce::h904fcd0dbdc71d4f () at std::syscommon::thread::startthread () at 0x00005555555f83b6 in std::sys::unix::thread::Thread::new::threadstart () at 0x00007ffff7f73a9d in startthread () from 0x00007ffff7e89a43 in clone () from While for another segfault the backtrace is instead: libinko::runtimepanic::displaypanic (process=0x7f, message="ObjectValue::asblock() called on a non block object") at 0x0000555555568baa in libinko::vm::machine::Machine::panic (self=0x55555567d8c0, process=0x7ffff71f8bd0, message="") at libinko::vm::machine::Machine::runwitherrorhandling (self=0x55555567d8c0, process=0x7ffff71f8bd0) at 0x00005555555c88a4 in libinko::vm::machine::Machine::startprimarythreads::{{closure}} (process=...) at <libinko::pool::PoolInner<T::process (self=<optimized out, index=4, closure=0x7ffff71f8c60) at 0x00005555555b739d in <libinko::pool::Pool<T::run::{{closure}} () at std::syscommon::backtrace::rustbeginshortbacktrace (f=...) at 0x00005555555cb0dc in std::thread::Builder::spawn::{{closure}}::{{closure}} () at <std::panic::AssertUnwindSafe<Fas core::ops::function::FnOnce<()::callonce (self=..., args=<optimized out) at std::panicking::try::docall (data=<optimized out) at 0x0000555555618a3a in rustmaybecatchpanic () at 0x00005555555ba39b in std::panicking::try (f=...) at std::panic::catchunwind (f=...) at std::thread::Builder::spawn::{{closure}} () at <F as alloc::boxed::FnBox<A::callbox (self=0x55555567e5d0, args=<optimized out) at 0x00005555556090db in LTdynLTCOutputRu20u20aGTasLTGT::callonce::h904fcd0dbdc71d4f () at std::syscommon::thread::startthread () at 0x00005555555f83b6 in std::sys::unix::thread::Thread::new::threadstart () at 0x00007ffff7f73a9d in startthread () from 0x00007ffff7e89a43 in clone () from And a third segfault: 0x0000555555568d84 in libinko::vm::machine::Machine::run (self=<optimized out, process=<optimized out) at 0x0000555555568b7f in libinko::vm::machine::Machine::runwitherrorhandling (self=0x55555567d8c0, process=0x7ffff6cf3bd0) at 0x00005555555c88a4 in libinko::vm::machine::Machine::startprimarythreads::{{closure}} (process=...) at <libinko::pool::PoolInner<T::process (self=<optimized out, index=10, closure=0x7ffff6cf3c60) at 0x00005555555b739d in <libinko::pool::Pool<T::run::{{closure}} () at std::syscommon::backtrace::rustbeginshortbacktrace (f=...) at 0x00005555555cb0dc in std::thread::Builder::spawn::{{closure}}::{{closure}} () at <std::panic::AssertUnwindSafe<Fas core::ops::function::FnOnce<()::callonce (self=..., args=<optimized out) at std::panicking::try::docall (data=<optimized out) at 0x0000555555618a3a in rustmaybecatchpanic () at 0x00005555555ba39b in std::panicking::try (f=...) at std::panic::catchunwind (f=...) at std::thread::Builder::spawn::{{closure}} () at <F as alloc::boxed::FnBox<A::callbox (self=0x55555567f590, args=<optimized out) at 0x00005555556090db in LTdynLTCOutputRu20u20aGTasLTGT::callonce::h904fcd0dbdc71d4f () at std::syscommon::thread::startthread () at 0x00005555555f83b6 in std::sys::unix::thread::Thread::new::threadstart () at 0x00007ffff7f73a9d in startthread () from 0x00007ffff7e89a43 in clone () from In case of the last segfault, it seems certain local variables that are used are NULL pointers, when this should be impossible. Debugging this in GDB proves to be quite difficult, as a variety of variables are reported as <optimized outeven when debugging symbols are included. For example, for the last backtrace the output of "info locals" is: The VM test suite passes, even when running . I'm wondering if perhaps code is optimised in the wrong way, and this is somehow not triggered in the test suite (certainly possible, code coverage is not 100%). Reproducing this is a bit weird. If we leave the code as-is, the segmentation faults rarely occur, instead the VM panics with the following: Stack trace (the most recent call comes last): 0: "", line 324, in "<block" 1: "", line 281, in "<lambda" 2: "", line 220, in "run" Process 1 panicked: ObjectValue::as_block() called on a non block object However, if we apply the following patch things will start to segfault really quick: Note that the last command requires Ruby 2.3 or newer. This will run the test suite of the standard library, which is where all the crashes happen rather frequently (probably because they run much more than the VM's own test suite).
There's a lot of questionable unsafe code in that project. For example, should really be marked as (same goes for . It's very likely that the cause of the issue is preexisting UB due to misuse of .
Simply marking functions as to my knowledge doesn't influence the compiler's code generation. Many of the unsafe operations are safe in practise, as they will only fail if (for example) the input bytecode is invalid. Tagging them as does nothing but make the code significantly more verbose. The various operations in the bytecode parser are there to convert sequences of bytes into particular types. To the best of my knowledge there is no stable way of converting a into say a , requiring you to use instead. The closest is but this: nightly the platform's endianness, instead of allowing you to specify which to use.
Yes, but marking a function as requires all call sites to opt into that unsafety, and the functions I mentioned all assume that their arguments are valid (and cause UB if not). This additional constraint on the arguments is expressed by marking the function and documenting the safety constraints. Check out the crate.
This wouldn't achieve much, as a lot would have to be marked unsafe as VMs and GCs inherently perform many operations Rust may consider unsafe. There are a few cases where I want to get rid of more risky code (e.g. the borrowing of you highlighted), but all of this feels rather offtopic since it's very project specific. I avoided it in the past because it also just used , so it didn't feel like much of an improvement. It appears this is no longer the case, so I'll take another look.
To better understand what is going on I dumped the MIR output using and (2018-08-18). After replacing a bunch of SHA hashes with placeholders (to reduce diff noise), I ended up with the exact same MIR before and after. Next I'll try LLVM IR.
It appears the LLVM IR uses a different order for things when using the different nightly versions. This generates a lot of diff noise, so I'm not sure how useful this information will be. I tried to clean it up a bit by replacing various SHA hashes with placeholders, but even then there is a lot of noise.
The ASM output was a bit easier to clean up, but there are still lots of differences. To be exact, reports: Again this is probably because of ordering. The new version has 99 lines more, but I'm really not sure if that's expected or not.
Using I found that nightly-2018-08-19 is the first version that introduces the regression, while nightly-2018-08-18 works fine. The commits included into the 18th nightly can be found at (if I got things right at least).
This hints at , which can change the behaviour of any latent UB, or , since your project uses in a few places.
I don't fully understand the first mentioned PR. What would I need to do to verify if that has any impact? For the second one, I'll see if I can somehow get my code to use a version of from before that PR, to see if it affects anything.
A custom version of probably isn't going to work. For example, I'm using Rayon which only implements its traits for the default :<
I ended up replacing all my uses of with to test if the VecDeque changes are to blame. I used LinkedList because its API is closest to VecDeque, making it easier to test things. When using LinkedList, my code runs fine. This suggests that the changes in may have somehow broken things. cc
The first PR changes details about the generated code that is passed to LLVM. In fact, it just removes unneeded information. The easiest way to check if it's at fault is to manually build the compiler from with the PR reverted. If the problem stops happening, it's at fault (or more likely, UB in your code is). It would also be helpful if you could reduce the code as much as possible so that we can get a few hundred line example that shows the issue.
Specifically, the patch I applied is:
I narrowed this down to the following file being broken by the changes: If I switch back to for this file, my code breaks. There are two possibilities here: was a bug before in my code that only somehow didn't get triggered before VecDeque changes themselves break things The Mailbox data structure has multiple threads accessing it concurrently. This is synchronised using a mutex of type (provided by parking_lot). The structure is broken up into essentially two parts: a synchronised external part, and an unsynchronised internal part. The internal part is only used by a single thread, while multiple threads can compete for the external part. For this reason the mutex is separate, so we don't need to synchronise both. Edit: I forgot, this is synchronised earlier on, preventing concurrent writes. I'll see if I can refactor this code a bit so it's safer, that way we can at least rule out my own code being the problem (or not).
Upon closer inspection, the VecDeque changes specifically affect , and I happen to use that in my Mailbox structure. Specifically, I have the following: I don't remember why this is using since is also a , probably I wrote this when not paying attention. Regardless, changing this to doesn't make any difference regarding the errors. Next I changed this code to the following: This is based on the old implementation of , which was: This then produces output like: This is a bit odd considering the implementation is the same as before, but at least now part of the program is running.
There was another place where I relied on . After also replacing that with the pattern my code now runs fine again, both on nightly and stable Rust. This suggests the failure is definitely due to the changes made to . Now to see if I can get this in a standalone example.
Good news: Thanks to the great investigation of and I think I found the reason for the segfault: the new calls which unsoundly creates mutable slices to uninitialized memory. I will try to send a fixing PR tomorrow.
Aha, thanks! I was looking at the code to see how I might be able to trigger this problem, but I hadn't made my way to just yet. Looking at the code, I'm not sure exactly how to trigger this reliably.
That could be it, nice find!
The problem AFAIK is that this function is triggering Undefined Behavior, and so can cause the program to segfault randomly, and so it may be difficult to reproduce it reliably.
I can verify that this is because of my PR. Minimal example that segfaults for me: And the reason for the segfault is apparent when run with simpler types This will print and continue in such a loop forever, printing the buffer over and over again. This happens because of line hitting an edge case when that causes head to be placed outside of the buffer. This can be fixed by changing that line to I think that is the only thing that is causing these segfaults.

#[inline] #[stable(feature = "append", since = "1.4.0")] pub fn append(&mut self, other: &mut Self) { <del> // Copies all values from `src_slice` to the start of `dst_slice`. unsafe fn copy_whole_slice<T>(src_slice: &[T], dst_slice: &mut [T]) { let len = src_slice.len(); ptr::copy_nonoverlapping(src_slice.as_ptr(), dst_slice[..len].as_mut_ptr(), len); } let src_total = other.len(); // Guarantees there is space in `self` for `other`. self.reserve(src_total); self.head = { let original_head = self.head; // The goal is to copy all values from `other` into `self`. To avoid any // mismatch, all valid values in `other` are retrieved... let (src_high, src_low) = other.as_slices(); // and unoccupied parts of self are retrieved. let (dst_high, dst_low) = unsafe { self.unused_as_mut_slices() }; // Then all that is needed is to copy all values from // src (src_high and src_low) to dst (dst_high and dst_low). // // other [o o o . . . . . o o o o] //       [5 6 7]         [1 2 3 4] //       src_low         src_high // // self  [. . . . . . o o o o . .] //       [3 4 5 6 7 .]       [1 2] //       dst_low             dst_high // // Values are not copied one by one but as slices in `copy_whole_slice`. // What slices are used depends on various properties of src and dst. // There are 6 cases in total: //     1. `src` is contiguous and fits in dst_high //     2. `src` is contiguous and does not fit in dst_high //     3. `src` is discontiguous and fits in dst_high //     4. `src` is discontiguous and does not fit in dst_high //        + src_high is smaller than dst_high //     5. `src` is discontiguous and does not fit in dst_high //        + dst_high is smaller than src_high //     6. `src` is discontiguous and does not fit in dst_high //        + dst_high is the same size as src_high let src_contiguous = src_low.is_empty(); let dst_high_fits_src = dst_high.len() >= src_total; match (src_contiguous, dst_high_fits_src) { (true, true) => { // 1. // other [. . . o o o . . . . . .] //       []    [1 1 1] // // self  [. o o o o o . . . . . .] //       [.]         [1 1 1 . . .] unsafe { copy_whole_slice(src_high, dst_high); } original_head + src_total } (true, false) => { // 2. // other [. . . o o o o o . . . .] //       []    [1 1 2 2 2] // // self  [. . . . . . . o o o . .] //       [2 2 2 . . . .]     [1 1] let (src_1, src_2) = src_high.split_at(dst_high.len()); unsafe { copy_whole_slice(src_1, dst_high); copy_whole_slice(src_2, dst_low); } src_total - dst_high.len() } (false, true) => { // 3. // other [o o . . . . . . . o o o] //       [2 2]             [1 1 1] // // self  [. o o . . . . . . . . .] //       [.]   [1 1 1 2 2 . . . .] let (dst_1, dst_2) = dst_high.split_at_mut(src_high.len()); unsafe { copy_whole_slice(src_high, dst_1); copy_whole_slice(src_low, dst_2); } original_head + src_total } (false, false) => { if src_high.len() < dst_high.len() { // 4. // other [o o o . . . . . . o o o] //       [2 3 3]           [1 1 1] // // self  [. . . . . . o o . . . .] //       [3 3 . . . .]   [1 1 1 2] let (dst_1, dst_2) = dst_high.split_at_mut(src_high.len()); let (src_2, src_3) = src_low.split_at(dst_2.len()); unsafe { copy_whole_slice(src_high, dst_1); copy_whole_slice(src_2, dst_2); copy_whole_slice(src_3, dst_low); } src_3.len() } else if src_high.len() > dst_high.len() { // 5. // other [o o o . . . . . o o o o] //       [3 3 3]         [1 1 2 2] // // self  [. . . . . . o o o o . .] //       [2 2 3 3 3 .]       [1 1] let (src_1, src_2) = src_high.split_at(dst_high.len()); let (dst_2, dst_3) = dst_low.split_at_mut(src_2.len()); unsafe { copy_whole_slice(src_1, dst_high); copy_whole_slice(src_2, dst_2); copy_whole_slice(src_low, dst_3); } dst_2.len() + src_low.len() } else { // 6. // other [o o . . . . . . . o o o] //       [2 2]             [1 1 1] // // self  [. . . . . . . o o . . .] //       [2 2 . . . . .]   [1 1 1] unsafe { copy_whole_slice(src_high, dst_high); copy_whole_slice(src_low, dst_low); } src_low.len() } } } }; // Some values now exist in both `other` and `self` but are made inaccessible in `other`. other.tail = other.head; </del> <ins> // naive impl self.extend(other.drain(..)); </ins> } /// Retains only the elements specified by the predicate.


Question:
a programming language that I am working on, and the VM is written in Rust. Up until Rust nightly 2018-08-17, everything works fine. Starting with the nightly from the 17th, I'm observing various crashes and different program behaviour. For example: On Windows it will either fail with a , or (more on this in a moment). On Linux . Note that the funny segfault output is because the command is started with Ruby, and Ruby installs its own segmentation fault handler. Locally it will usually fail with the same runtime error as observed in Windows above, but sometimes it will segfault. Sometimes it will panic because certain operations are performed using NULL pointers where this is not expected. The last nightly that did not suffer from these problems was Rust 2018-08-16. Stable Rust also works fine. When the segmentation faults happen, they are usually in different places. For example, for one segmentation fault the backtrace is as follows: 0x00007ffff7e12763 in intmalloc () from 0x00007ffff7e13ada in malloc () from 0x0000555555568e6b in alloc::alloc::alloc (layout=...) at <libinko::chunk::Chunk<T::new (capacity=3) at libinko::register::Register::new (amount=3) at libinko::executioncontext::ExecutionContext::fromblock (block=0x7fffdc0971e0, returnregister=Some = {...}) at libinko::vm::machine::Machine::run (self=<optimized out, process=<optimized out) at 0x0000555555568b7f in libinko::vm::machine::Machine::runwitherrorhandling (self=0x55555567d8c0, process=0x7ffff75fcbd0) at 0x00005555555c88a4 in libinko::vm::machine::Machine::startprimarythreads::{{closure}} (process=...) at <libinko::pool::PoolInner<T::process (self=<optimized out, index=0, closure=0x7ffff75fcc60) at 0x00005555555b739d in <libinko::pool::Pool<T::run::{{closure}} () at std::syscommon::backtrace::rustbeginshortbacktrace (f=...) at 0x00005555555cb0dc in std::thread::Builder::spawn::{{closure}}::{{closure}} () at <std::panic::AssertUnwindSafe<Fas core::ops::function::FnOnce<()::callonce (self=..., args=<optimized out) at std::panicking::try::docall (data=<optimized out) at 0x0000555555618a3a in rustmaybecatchpanic () at 0x00005555555ba39b in std::panicking::try (f=...) at std::panic::catchunwind (f=...) at std::thread::Builder::spawn::{{closure}} () at <F as alloc::boxed::FnBox<A::callbox (self=0x55555567db50, args=<optimized out) at 0x00005555556090db in LTdynLTCOutputRu20u20aGTasLTGT::callonce::h904fcd0dbdc71d4f () at std::syscommon::thread::startthread () at 0x00005555555f83b6 in std::sys::unix::thread::Thread::new::threadstart () at 0x00007ffff7f73a9d in startthread () from 0x00007ffff7e89a43 in clone () from While for another segfault the backtrace is instead: libinko::runtimepanic::displaypanic (process=0x7f, message="ObjectValue::asblock() called on a non block object") at 0x0000555555568baa in libinko::vm::machine::Machine::panic (self=0x55555567d8c0, process=0x7ffff71f8bd0, message="") at libinko::vm::machine::Machine::runwitherrorhandling (self=0x55555567d8c0, process=0x7ffff71f8bd0) at 0x00005555555c88a4 in libinko::vm::machine::Machine::startprimarythreads::{{closure}} (process=...) at <libinko::pool::PoolInner<T::process (self=<optimized out, index=4, closure=0x7ffff71f8c60) at 0x00005555555b739d in <libinko::pool::Pool<T::run::{{closure}} () at std::syscommon::backtrace::rustbeginshortbacktrace (f=...) at 0x00005555555cb0dc in std::thread::Builder::spawn::{{closure}}::{{closure}} () at <std::panic::AssertUnwindSafe<Fas core::ops::function::FnOnce<()::callonce (self=..., args=<optimized out) at std::panicking::try::docall (data=<optimized out) at 0x0000555555618a3a in rustmaybecatchpanic () at 0x00005555555ba39b in std::panicking::try (f=...) at std::panic::catchunwind (f=...) at std::thread::Builder::spawn::{{closure}} () at <F as alloc::boxed::FnBox<A::callbox (self=0x55555567e5d0, args=<optimized out) at 0x00005555556090db in LTdynLTCOutputRu20u20aGTasLTGT::callonce::h904fcd0dbdc71d4f () at std::syscommon::thread::startthread () at 0x00005555555f83b6 in std::sys::unix::thread::Thread::new::threadstart () at 0x00007ffff7f73a9d in startthread () from 0x00007ffff7e89a43 in clone () from And a third segfault: 0x0000555555568d84 in libinko::vm::machine::Machine::run (self=<optimized out, process=<optimized out) at 0x0000555555568b7f in libinko::vm::machine::Machine::runwitherrorhandling (self=0x55555567d8c0, process=0x7ffff6cf3bd0) at 0x00005555555c88a4 in libinko::vm::machine::Machine::startprimarythreads::{{closure}} (process=...) at <libinko::pool::PoolInner<T::process (self=<optimized out, index=10, closure=0x7ffff6cf3c60) at 0x00005555555b739d in <libinko::pool::Pool<T::run::{{closure}} () at std::syscommon::backtrace::rustbeginshortbacktrace (f=...) at 0x00005555555cb0dc in std::thread::Builder::spawn::{{closure}}::{{closure}} () at <std::panic::AssertUnwindSafe<Fas core::ops::function::FnOnce<()::callonce (self=..., args=<optimized out) at std::panicking::try::docall (data=<optimized out) at 0x0000555555618a3a in rustmaybecatchpanic () at 0x00005555555ba39b in std::panicking::try (f=...) at std::panic::catchunwind (f=...) at std::thread::Builder::spawn::{{closure}} () at <F as alloc::boxed::FnBox<A::callbox (self=0x55555567f590, args=<optimized out) at 0x00005555556090db in LTdynLTCOutputRu20u20aGTasLTGT::callonce::h904fcd0dbdc71d4f () at std::syscommon::thread::startthread () at 0x00005555555f83b6 in std::sys::unix::thread::Thread::new::threadstart () at 0x00007ffff7f73a9d in startthread () from 0x00007ffff7e89a43 in clone () from In case of the last segfault, it seems certain local variables that are used are NULL pointers, when this should be impossible. Debugging this in GDB proves to be quite difficult, as a variety of variables are reported as <optimized outeven when debugging symbols are included. For example, for the last backtrace the output of "info locals" is: The VM test suite passes, even when running . I'm wondering if perhaps code is optimised in the wrong way, and this is somehow not triggered in the test suite (certainly possible, code coverage is not 100%). Reproducing this is a bit weird. If we leave the code as-is, the segmentation faults rarely occur, instead the VM panics with the following: Stack trace (the most recent call comes last): 0: "", line 324, in "<block" 1: "", line 281, in "<lambda" 2: "", line 220, in "run" Process 1 panicked: ObjectValue::as_block() called on a non block object However, if we apply the following patch things will start to segfault really quick: Note that the last command requires Ruby 2.3 or newer. This will run the test suite of the standard library, which is where all the crashes happen rather frequently (probably because they run much more than the VM's own test suite).
There's a lot of questionable unsafe code in that project. For example, should really be marked as (same goes for . It's very likely that the cause of the issue is preexisting UB due to misuse of .
Simply marking functions as to my knowledge doesn't influence the compiler's code generation. Many of the unsafe operations are safe in practise, as they will only fail if (for example) the input bytecode is invalid. Tagging them as does nothing but make the code significantly more verbose. The various operations in the bytecode parser are there to convert sequences of bytes into particular types. To the best of my knowledge there is no stable way of converting a into say a , requiring you to use instead. The closest is but this: nightly the platform's endianness, instead of allowing you to specify which to use.
Yes, but marking a function as requires all call sites to opt into that unsafety, and the functions I mentioned all assume that their arguments are valid (and cause UB if not). This additional constraint on the arguments is expressed by marking the function and documenting the safety constraints. Check out the crate.
This wouldn't achieve much, as a lot would have to be marked unsafe as VMs and GCs inherently perform many operations Rust may consider unsafe. There are a few cases where I want to get rid of more risky code (e.g. the borrowing of you highlighted), but all of this feels rather offtopic since it's very project specific. I avoided it in the past because it also just used , so it didn't feel like much of an improvement. It appears this is no longer the case, so I'll take another look.
To better understand what is going on I dumped the MIR output using and (2018-08-18). After replacing a bunch of SHA hashes with placeholders (to reduce diff noise), I ended up with the exact same MIR before and after. Next I'll try LLVM IR.
It appears the LLVM IR uses a different order for things when using the different nightly versions. This generates a lot of diff noise, so I'm not sure how useful this information will be. I tried to clean it up a bit by replacing various SHA hashes with placeholders, but even then there is a lot of noise.
The ASM output was a bit easier to clean up, but there are still lots of differences. To be exact, reports: Again this is probably because of ordering. The new version has 99 lines more, but I'm really not sure if that's expected or not.
Using I found that nightly-2018-08-19 is the first version that introduces the regression, while nightly-2018-08-18 works fine. The commits included into the 18th nightly can be found at (if I got things right at least).
This hints at , which can change the behaviour of any latent UB, or , since your project uses in a few places.
I don't fully understand the first mentioned PR. What would I need to do to verify if that has any impact? For the second one, I'll see if I can somehow get my code to use a version of from before that PR, to see if it affects anything.
A custom version of probably isn't going to work. For example, I'm using Rayon which only implements its traits for the default :<
I ended up replacing all my uses of with to test if the VecDeque changes are to blame. I used LinkedList because its API is closest to VecDeque, making it easier to test things. When using LinkedList, my code runs fine. This suggests that the changes in may have somehow broken things. cc
The first PR changes details about the generated code that is passed to LLVM. In fact, it just removes unneeded information. The easiest way to check if it's at fault is to manually build the compiler from with the PR reverted. If the problem stops happening, it's at fault (or more likely, UB in your code is). It would also be helpful if you could reduce the code as much as possible so that we can get a few hundred line example that shows the issue.
Specifically, the patch I applied is:
I narrowed this down to the following file being broken by the changes: If I switch back to for this file, my code breaks. There are two possibilities here: was a bug before in my code that only somehow didn't get triggered before VecDeque changes themselves break things The Mailbox data structure has multiple threads accessing it concurrently. This is synchronised using a mutex of type (provided by parking_lot). The structure is broken up into essentially two parts: a synchronised external part, and an unsynchronised internal part. The internal part is only used by a single thread, while multiple threads can compete for the external part. For this reason the mutex is separate, so we don't need to synchronise both. Edit: I forgot, this is synchronised earlier on, preventing concurrent writes. I'll see if I can refactor this code a bit so it's safer, that way we can at least rule out my own code being the problem (or not).
Upon closer inspection, the VecDeque changes specifically affect , and I happen to use that in my Mailbox structure. Specifically, I have the following: I don't remember why this is using since is also a , probably I wrote this when not paying attention. Regardless, changing this to doesn't make any difference regarding the errors. Next I changed this code to the following: This is based on the old implementation of , which was: This then produces output like: This is a bit odd considering the implementation is the same as before, but at least now part of the program is running.
There was another place where I relied on . After also replacing that with the pattern my code now runs fine again, both on nightly and stable Rust. This suggests the failure is definitely due to the changes made to . Now to see if I can get this in a standalone example.
Good news: Thanks to the great investigation of and I think I found the reason for the segfault: the new calls which unsoundly creates mutable slices to uninitialized memory. I will try to send a fixing PR tomorrow.
Aha, thanks! I was looking at the code to see how I might be able to trigger this problem, but I hadn't made my way to just yet. Looking at the code, I'm not sure exactly how to trigger this reliably.
That could be it, nice find!
The problem AFAIK is that this function is triggering Undefined Behavior, and so can cause the program to segfault randomly, and so it may be difficult to reproduce it reliably.
I can verify that this is because of my PR. Minimal example that segfaults for me: And the reason for the segfault is apparent when run with simpler types This will print and continue in such a loop forever, printing the buffer over and over again. This happens because of line hitting an edge case when that causes head to be placed outside of the buffer. This can be fixed by changing that line to I think that is the only thing that is causing these segfaults.

#[stable(feature = "collection_debug", since = "1.17.0")] impl<'a, T: 'a + fmt::Debug> fmt::Debug for Iter<'a, T> { fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { <ins> let (front, back) = RingSlices::ring_slices(self.ring, self.head, self.tail); </ins> f.debug_tuple("Iter") <del> .field(&self.ring) .field(&self.tail) .field(&self.head) .finish() </del> <ins> .field(&front) .field(&back) .finish() </ins> } }


Question:
a programming language that I am working on, and the VM is written in Rust. Up until Rust nightly 2018-08-17, everything works fine. Starting with the nightly from the 17th, I'm observing various crashes and different program behaviour. For example: On Windows it will either fail with a , or (more on this in a moment). On Linux . Note that the funny segfault output is because the command is started with Ruby, and Ruby installs its own segmentation fault handler. Locally it will usually fail with the same runtime error as observed in Windows above, but sometimes it will segfault. Sometimes it will panic because certain operations are performed using NULL pointers where this is not expected. The last nightly that did not suffer from these problems was Rust 2018-08-16. Stable Rust also works fine. When the segmentation faults happen, they are usually in different places. For example, for one segmentation fault the backtrace is as follows: 0x00007ffff7e12763 in intmalloc () from 0x00007ffff7e13ada in malloc () from 0x0000555555568e6b in alloc::alloc::alloc (layout=...) at <libinko::chunk::Chunk<T::new (capacity=3) at libinko::register::Register::new (amount=3) at libinko::executioncontext::ExecutionContext::fromblock (block=0x7fffdc0971e0, returnregister=Some = {...}) at libinko::vm::machine::Machine::run (self=<optimized out, process=<optimized out) at 0x0000555555568b7f in libinko::vm::machine::Machine::runwitherrorhandling (self=0x55555567d8c0, process=0x7ffff75fcbd0) at 0x00005555555c88a4 in libinko::vm::machine::Machine::startprimarythreads::{{closure}} (process=...) at <libinko::pool::PoolInner<T::process (self=<optimized out, index=0, closure=0x7ffff75fcc60) at 0x00005555555b739d in <libinko::pool::Pool<T::run::{{closure}} () at std::syscommon::backtrace::rustbeginshortbacktrace (f=...) at 0x00005555555cb0dc in std::thread::Builder::spawn::{{closure}}::{{closure}} () at <std::panic::AssertUnwindSafe<Fas core::ops::function::FnOnce<()::callonce (self=..., args=<optimized out) at std::panicking::try::docall (data=<optimized out) at 0x0000555555618a3a in rustmaybecatchpanic () at 0x00005555555ba39b in std::panicking::try (f=...) at std::panic::catchunwind (f=...) at std::thread::Builder::spawn::{{closure}} () at <F as alloc::boxed::FnBox<A::callbox (self=0x55555567db50, args=<optimized out) at 0x00005555556090db in LTdynLTCOutputRu20u20aGTasLTGT::callonce::h904fcd0dbdc71d4f () at std::syscommon::thread::startthread () at 0x00005555555f83b6 in std::sys::unix::thread::Thread::new::threadstart () at 0x00007ffff7f73a9d in startthread () from 0x00007ffff7e89a43 in clone () from While for another segfault the backtrace is instead: libinko::runtimepanic::displaypanic (process=0x7f, message="ObjectValue::asblock() called on a non block object") at 0x0000555555568baa in libinko::vm::machine::Machine::panic (self=0x55555567d8c0, process=0x7ffff71f8bd0, message="") at libinko::vm::machine::Machine::runwitherrorhandling (self=0x55555567d8c0, process=0x7ffff71f8bd0) at 0x00005555555c88a4 in libinko::vm::machine::Machine::startprimarythreads::{{closure}} (process=...) at <libinko::pool::PoolInner<T::process (self=<optimized out, index=4, closure=0x7ffff71f8c60) at 0x00005555555b739d in <libinko::pool::Pool<T::run::{{closure}} () at std::syscommon::backtrace::rustbeginshortbacktrace (f=...) at 0x00005555555cb0dc in std::thread::Builder::spawn::{{closure}}::{{closure}} () at <std::panic::AssertUnwindSafe<Fas core::ops::function::FnOnce<()::callonce (self=..., args=<optimized out) at std::panicking::try::docall (data=<optimized out) at 0x0000555555618a3a in rustmaybecatchpanic () at 0x00005555555ba39b in std::panicking::try (f=...) at std::panic::catchunwind (f=...) at std::thread::Builder::spawn::{{closure}} () at <F as alloc::boxed::FnBox<A::callbox (self=0x55555567e5d0, args=<optimized out) at 0x00005555556090db in LTdynLTCOutputRu20u20aGTasLTGT::callonce::h904fcd0dbdc71d4f () at std::syscommon::thread::startthread () at 0x00005555555f83b6 in std::sys::unix::thread::Thread::new::threadstart () at 0x00007ffff7f73a9d in startthread () from 0x00007ffff7e89a43 in clone () from And a third segfault: 0x0000555555568d84 in libinko::vm::machine::Machine::run (self=<optimized out, process=<optimized out) at 0x0000555555568b7f in libinko::vm::machine::Machine::runwitherrorhandling (self=0x55555567d8c0, process=0x7ffff6cf3bd0) at 0x00005555555c88a4 in libinko::vm::machine::Machine::startprimarythreads::{{closure}} (process=...) at <libinko::pool::PoolInner<T::process (self=<optimized out, index=10, closure=0x7ffff6cf3c60) at 0x00005555555b739d in <libinko::pool::Pool<T::run::{{closure}} () at std::syscommon::backtrace::rustbeginshortbacktrace (f=...) at 0x00005555555cb0dc in std::thread::Builder::spawn::{{closure}}::{{closure}} () at <std::panic::AssertUnwindSafe<Fas core::ops::function::FnOnce<()::callonce (self=..., args=<optimized out) at std::panicking::try::docall (data=<optimized out) at 0x0000555555618a3a in rustmaybecatchpanic () at 0x00005555555ba39b in std::panicking::try (f=...) at std::panic::catchunwind (f=...) at std::thread::Builder::spawn::{{closure}} () at <F as alloc::boxed::FnBox<A::callbox (self=0x55555567f590, args=<optimized out) at 0x00005555556090db in LTdynLTCOutputRu20u20aGTasLTGT::callonce::h904fcd0dbdc71d4f () at std::syscommon::thread::startthread () at 0x00005555555f83b6 in std::sys::unix::thread::Thread::new::threadstart () at 0x00007ffff7f73a9d in startthread () from 0x00007ffff7e89a43 in clone () from In case of the last segfault, it seems certain local variables that are used are NULL pointers, when this should be impossible. Debugging this in GDB proves to be quite difficult, as a variety of variables are reported as <optimized outeven when debugging symbols are included. For example, for the last backtrace the output of "info locals" is: The VM test suite passes, even when running . I'm wondering if perhaps code is optimised in the wrong way, and this is somehow not triggered in the test suite (certainly possible, code coverage is not 100%). Reproducing this is a bit weird. If we leave the code as-is, the segmentation faults rarely occur, instead the VM panics with the following: Stack trace (the most recent call comes last): 0: "", line 324, in "<block" 1: "", line 281, in "<lambda" 2: "", line 220, in "run" Process 1 panicked: ObjectValue::as_block() called on a non block object However, if we apply the following patch things will start to segfault really quick: Note that the last command requires Ruby 2.3 or newer. This will run the test suite of the standard library, which is where all the crashes happen rather frequently (probably because they run much more than the VM's own test suite).
There's a lot of questionable unsafe code in that project. For example, should really be marked as (same goes for . It's very likely that the cause of the issue is preexisting UB due to misuse of .
Simply marking functions as to my knowledge doesn't influence the compiler's code generation. Many of the unsafe operations are safe in practise, as they will only fail if (for example) the input bytecode is invalid. Tagging them as does nothing but make the code significantly more verbose. The various operations in the bytecode parser are there to convert sequences of bytes into particular types. To the best of my knowledge there is no stable way of converting a into say a , requiring you to use instead. The closest is but this: nightly the platform's endianness, instead of allowing you to specify which to use.
Yes, but marking a function as requires all call sites to opt into that unsafety, and the functions I mentioned all assume that their arguments are valid (and cause UB if not). This additional constraint on the arguments is expressed by marking the function and documenting the safety constraints. Check out the crate.
This wouldn't achieve much, as a lot would have to be marked unsafe as VMs and GCs inherently perform many operations Rust may consider unsafe. There are a few cases where I want to get rid of more risky code (e.g. the borrowing of you highlighted), but all of this feels rather offtopic since it's very project specific. I avoided it in the past because it also just used , so it didn't feel like much of an improvement. It appears this is no longer the case, so I'll take another look.
To better understand what is going on I dumped the MIR output using and (2018-08-18). After replacing a bunch of SHA hashes with placeholders (to reduce diff noise), I ended up with the exact same MIR before and after. Next I'll try LLVM IR.
It appears the LLVM IR uses a different order for things when using the different nightly versions. This generates a lot of diff noise, so I'm not sure how useful this information will be. I tried to clean it up a bit by replacing various SHA hashes with placeholders, but even then there is a lot of noise.
The ASM output was a bit easier to clean up, but there are still lots of differences. To be exact, reports: Again this is probably because of ordering. The new version has 99 lines more, but I'm really not sure if that's expected or not.
Using I found that nightly-2018-08-19 is the first version that introduces the regression, while nightly-2018-08-18 works fine. The commits included into the 18th nightly can be found at (if I got things right at least).
This hints at , which can change the behaviour of any latent UB, or , since your project uses in a few places.
I don't fully understand the first mentioned PR. What would I need to do to verify if that has any impact? For the second one, I'll see if I can somehow get my code to use a version of from before that PR, to see if it affects anything.
A custom version of probably isn't going to work. For example, I'm using Rayon which only implements its traits for the default :<
I ended up replacing all my uses of with to test if the VecDeque changes are to blame. I used LinkedList because its API is closest to VecDeque, making it easier to test things. When using LinkedList, my code runs fine. This suggests that the changes in may have somehow broken things. cc
The first PR changes details about the generated code that is passed to LLVM. In fact, it just removes unneeded information. The easiest way to check if it's at fault is to manually build the compiler from with the PR reverted. If the problem stops happening, it's at fault (or more likely, UB in your code is). It would also be helpful if you could reduce the code as much as possible so that we can get a few hundred line example that shows the issue.
Specifically, the patch I applied is:
I narrowed this down to the following file being broken by the changes: If I switch back to for this file, my code breaks. There are two possibilities here: was a bug before in my code that only somehow didn't get triggered before VecDeque changes themselves break things The Mailbox data structure has multiple threads accessing it concurrently. This is synchronised using a mutex of type (provided by parking_lot). The structure is broken up into essentially two parts: a synchronised external part, and an unsynchronised internal part. The internal part is only used by a single thread, while multiple threads can compete for the external part. For this reason the mutex is separate, so we don't need to synchronise both. Edit: I forgot, this is synchronised earlier on, preventing concurrent writes. I'll see if I can refactor this code a bit so it's safer, that way we can at least rule out my own code being the problem (or not).
Upon closer inspection, the VecDeque changes specifically affect , and I happen to use that in my Mailbox structure. Specifically, I have the following: I don't remember why this is using since is also a , probably I wrote this when not paying attention. Regardless, changing this to doesn't make any difference regarding the errors. Next I changed this code to the following: This is based on the old implementation of , which was: This then produces output like: This is a bit odd considering the implementation is the same as before, but at least now part of the program is running.
There was another place where I relied on . After also replacing that with the pattern my code now runs fine again, both on nightly and stable Rust. This suggests the failure is definitely due to the changes made to . Now to see if I can get this in a standalone example.
Good news: Thanks to the great investigation of and I think I found the reason for the segfault: the new calls which unsoundly creates mutable slices to uninitialized memory. I will try to send a fixing PR tomorrow.
Aha, thanks! I was looking at the code to see how I might be able to trigger this problem, but I hadn't made my way to just yet. Looking at the code, I'm not sure exactly how to trigger this reliably.
That could be it, nice find!
The problem AFAIK is that this function is triggering Undefined Behavior, and so can cause the program to segfault randomly, and so it may be difficult to reproduce it reliably.
I can verify that this is because of my PR. Minimal example that segfaults for me: And the reason for the segfault is apparent when run with simpler types This will print and continue in such a loop forever, printing the buffer over and over again. This happens because of line hitting an edge case when that causes head to be placed outside of the buffer. This can be fixed by changing that line to I think that is the only thing that is causing these segfaults.

#[stable(feature = "collection_debug", since = "1.17.0")] impl<'a, T: 'a + fmt::Debug> fmt::Debug for IterMut<'a, T> { fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { <ins> let (front, back) = RingSlices::ring_slices(&*self.ring, self.head, self.tail); </ins> f.debug_tuple("IterMut") <del> .field(&self.ring) .field(&self.tail) .field(&self.head) .finish() </del> <ins> .field(&front) .field(&back) .finish() </ins> } }


Question:
a programming language that I am working on, and the VM is written in Rust. Up until Rust nightly 2018-08-17, everything works fine. Starting with the nightly from the 17th, I'm observing various crashes and different program behaviour. For example: On Windows it will either fail with a , or (more on this in a moment). On Linux . Note that the funny segfault output is because the command is started with Ruby, and Ruby installs its own segmentation fault handler. Locally it will usually fail with the same runtime error as observed in Windows above, but sometimes it will segfault. Sometimes it will panic because certain operations are performed using NULL pointers where this is not expected. The last nightly that did not suffer from these problems was Rust 2018-08-16. Stable Rust also works fine. When the segmentation faults happen, they are usually in different places. For example, for one segmentation fault the backtrace is as follows: 0x00007ffff7e12763 in intmalloc () from 0x00007ffff7e13ada in malloc () from 0x0000555555568e6b in alloc::alloc::alloc (layout=...) at <libinko::chunk::Chunk<T::new (capacity=3) at libinko::register::Register::new (amount=3) at libinko::executioncontext::ExecutionContext::fromblock (block=0x7fffdc0971e0, returnregister=Some = {...}) at libinko::vm::machine::Machine::run (self=<optimized out, process=<optimized out) at 0x0000555555568b7f in libinko::vm::machine::Machine::runwitherrorhandling (self=0x55555567d8c0, process=0x7ffff75fcbd0) at 0x00005555555c88a4 in libinko::vm::machine::Machine::startprimarythreads::{{closure}} (process=...) at <libinko::pool::PoolInner<T::process (self=<optimized out, index=0, closure=0x7ffff75fcc60) at 0x00005555555b739d in <libinko::pool::Pool<T::run::{{closure}} () at std::syscommon::backtrace::rustbeginshortbacktrace (f=...) at 0x00005555555cb0dc in std::thread::Builder::spawn::{{closure}}::{{closure}} () at <std::panic::AssertUnwindSafe<Fas core::ops::function::FnOnce<()::callonce (self=..., args=<optimized out) at std::panicking::try::docall (data=<optimized out) at 0x0000555555618a3a in rustmaybecatchpanic () at 0x00005555555ba39b in std::panicking::try (f=...) at std::panic::catchunwind (f=...) at std::thread::Builder::spawn::{{closure}} () at <F as alloc::boxed::FnBox<A::callbox (self=0x55555567db50, args=<optimized out) at 0x00005555556090db in LTdynLTCOutputRu20u20aGTasLTGT::callonce::h904fcd0dbdc71d4f () at std::syscommon::thread::startthread () at 0x00005555555f83b6 in std::sys::unix::thread::Thread::new::threadstart () at 0x00007ffff7f73a9d in startthread () from 0x00007ffff7e89a43 in clone () from While for another segfault the backtrace is instead: libinko::runtimepanic::displaypanic (process=0x7f, message="ObjectValue::asblock() called on a non block object") at 0x0000555555568baa in libinko::vm::machine::Machine::panic (self=0x55555567d8c0, process=0x7ffff71f8bd0, message="") at libinko::vm::machine::Machine::runwitherrorhandling (self=0x55555567d8c0, process=0x7ffff71f8bd0) at 0x00005555555c88a4 in libinko::vm::machine::Machine::startprimarythreads::{{closure}} (process=...) at <libinko::pool::PoolInner<T::process (self=<optimized out, index=4, closure=0x7ffff71f8c60) at 0x00005555555b739d in <libinko::pool::Pool<T::run::{{closure}} () at std::syscommon::backtrace::rustbeginshortbacktrace (f=...) at 0x00005555555cb0dc in std::thread::Builder::spawn::{{closure}}::{{closure}} () at <std::panic::AssertUnwindSafe<Fas core::ops::function::FnOnce<()::callonce (self=..., args=<optimized out) at std::panicking::try::docall (data=<optimized out) at 0x0000555555618a3a in rustmaybecatchpanic () at 0x00005555555ba39b in std::panicking::try (f=...) at std::panic::catchunwind (f=...) at std::thread::Builder::spawn::{{closure}} () at <F as alloc::boxed::FnBox<A::callbox (self=0x55555567e5d0, args=<optimized out) at 0x00005555556090db in LTdynLTCOutputRu20u20aGTasLTGT::callonce::h904fcd0dbdc71d4f () at std::syscommon::thread::startthread () at 0x00005555555f83b6 in std::sys::unix::thread::Thread::new::threadstart () at 0x00007ffff7f73a9d in startthread () from 0x00007ffff7e89a43 in clone () from And a third segfault: 0x0000555555568d84 in libinko::vm::machine::Machine::run (self=<optimized out, process=<optimized out) at 0x0000555555568b7f in libinko::vm::machine::Machine::runwitherrorhandling (self=0x55555567d8c0, process=0x7ffff6cf3bd0) at 0x00005555555c88a4 in libinko::vm::machine::Machine::startprimarythreads::{{closure}} (process=...) at <libinko::pool::PoolInner<T::process (self=<optimized out, index=10, closure=0x7ffff6cf3c60) at 0x00005555555b739d in <libinko::pool::Pool<T::run::{{closure}} () at std::syscommon::backtrace::rustbeginshortbacktrace (f=...) at 0x00005555555cb0dc in std::thread::Builder::spawn::{{closure}}::{{closure}} () at <std::panic::AssertUnwindSafe<Fas core::ops::function::FnOnce<()::callonce (self=..., args=<optimized out) at std::panicking::try::docall (data=<optimized out) at 0x0000555555618a3a in rustmaybecatchpanic () at 0x00005555555ba39b in std::panicking::try (f=...) at std::panic::catchunwind (f=...) at std::thread::Builder::spawn::{{closure}} () at <F as alloc::boxed::FnBox<A::callbox (self=0x55555567f590, args=<optimized out) at 0x00005555556090db in LTdynLTCOutputRu20u20aGTasLTGT::callonce::h904fcd0dbdc71d4f () at std::syscommon::thread::startthread () at 0x00005555555f83b6 in std::sys::unix::thread::Thread::new::threadstart () at 0x00007ffff7f73a9d in startthread () from 0x00007ffff7e89a43 in clone () from In case of the last segfault, it seems certain local variables that are used are NULL pointers, when this should be impossible. Debugging this in GDB proves to be quite difficult, as a variety of variables are reported as <optimized outeven when debugging symbols are included. For example, for the last backtrace the output of "info locals" is: The VM test suite passes, even when running . I'm wondering if perhaps code is optimised in the wrong way, and this is somehow not triggered in the test suite (certainly possible, code coverage is not 100%). Reproducing this is a bit weird. If we leave the code as-is, the segmentation faults rarely occur, instead the VM panics with the following: Stack trace (the most recent call comes last): 0: "", line 324, in "<block" 1: "", line 281, in "<lambda" 2: "", line 220, in "run" Process 1 panicked: ObjectValue::as_block() called on a non block object However, if we apply the following patch things will start to segfault really quick: Note that the last command requires Ruby 2.3 or newer. This will run the test suite of the standard library, which is where all the crashes happen rather frequently (probably because they run much more than the VM's own test suite).
There's a lot of questionable unsafe code in that project. For example, should really be marked as (same goes for . It's very likely that the cause of the issue is preexisting UB due to misuse of .
Simply marking functions as to my knowledge doesn't influence the compiler's code generation. Many of the unsafe operations are safe in practise, as they will only fail if (for example) the input bytecode is invalid. Tagging them as does nothing but make the code significantly more verbose. The various operations in the bytecode parser are there to convert sequences of bytes into particular types. To the best of my knowledge there is no stable way of converting a into say a , requiring you to use instead. The closest is but this: nightly the platform's endianness, instead of allowing you to specify which to use.
Yes, but marking a function as requires all call sites to opt into that unsafety, and the functions I mentioned all assume that their arguments are valid (and cause UB if not). This additional constraint on the arguments is expressed by marking the function and documenting the safety constraints. Check out the crate.
This wouldn't achieve much, as a lot would have to be marked unsafe as VMs and GCs inherently perform many operations Rust may consider unsafe. There are a few cases where I want to get rid of more risky code (e.g. the borrowing of you highlighted), but all of this feels rather offtopic since it's very project specific. I avoided it in the past because it also just used , so it didn't feel like much of an improvement. It appears this is no longer the case, so I'll take another look.
To better understand what is going on I dumped the MIR output using and (2018-08-18). After replacing a bunch of SHA hashes with placeholders (to reduce diff noise), I ended up with the exact same MIR before and after. Next I'll try LLVM IR.
It appears the LLVM IR uses a different order for things when using the different nightly versions. This generates a lot of diff noise, so I'm not sure how useful this information will be. I tried to clean it up a bit by replacing various SHA hashes with placeholders, but even then there is a lot of noise.
The ASM output was a bit easier to clean up, but there are still lots of differences. To be exact, reports: Again this is probably because of ordering. The new version has 99 lines more, but I'm really not sure if that's expected or not.
Using I found that nightly-2018-08-19 is the first version that introduces the regression, while nightly-2018-08-18 works fine. The commits included into the 18th nightly can be found at (if I got things right at least).
This hints at , which can change the behaviour of any latent UB, or , since your project uses in a few places.
I don't fully understand the first mentioned PR. What would I need to do to verify if that has any impact? For the second one, I'll see if I can somehow get my code to use a version of from before that PR, to see if it affects anything.
A custom version of probably isn't going to work. For example, I'm using Rayon which only implements its traits for the default :<
I ended up replacing all my uses of with to test if the VecDeque changes are to blame. I used LinkedList because its API is closest to VecDeque, making it easier to test things. When using LinkedList, my code runs fine. This suggests that the changes in may have somehow broken things. cc
The first PR changes details about the generated code that is passed to LLVM. In fact, it just removes unneeded information. The easiest way to check if it's at fault is to manually build the compiler from with the PR reverted. If the problem stops happening, it's at fault (or more likely, UB in your code is). It would also be helpful if you could reduce the code as much as possible so that we can get a few hundred line example that shows the issue.
Specifically, the patch I applied is:
I narrowed this down to the following file being broken by the changes: If I switch back to for this file, my code breaks. There are two possibilities here: was a bug before in my code that only somehow didn't get triggered before VecDeque changes themselves break things The Mailbox data structure has multiple threads accessing it concurrently. This is synchronised using a mutex of type (provided by parking_lot). The structure is broken up into essentially two parts: a synchronised external part, and an unsynchronised internal part. The internal part is only used by a single thread, while multiple threads can compete for the external part. For this reason the mutex is separate, so we don't need to synchronise both. Edit: I forgot, this is synchronised earlier on, preventing concurrent writes. I'll see if I can refactor this code a bit so it's safer, that way we can at least rule out my own code being the problem (or not).
Upon closer inspection, the VecDeque changes specifically affect , and I happen to use that in my Mailbox structure. Specifically, I have the following: I don't remember why this is using since is also a , probably I wrote this when not paying attention. Regardless, changing this to doesn't make any difference regarding the errors. Next I changed this code to the following: This is based on the old implementation of , which was: This then produces output like: This is a bit odd considering the implementation is the same as before, but at least now part of the program is running.
There was another place where I relied on . After also replacing that with the pattern my code now runs fine again, both on nightly and stable Rust. This suggests the failure is definitely due to the changes made to . Now to see if I can get this in a standalone example.
Good news: Thanks to the great investigation of and I think I found the reason for the segfault: the new calls which unsoundly creates mutable slices to uninitialized memory. I will try to send a fixing PR tomorrow.
Aha, thanks! I was looking at the code to see how I might be able to trigger this problem, but I hadn't made my way to just yet. Looking at the code, I'm not sure exactly how to trigger this reliably.
That could be it, nice find!
The problem AFAIK is that this function is triggering Undefined Behavior, and so can cause the program to segfault randomly, and so it may be difficult to reproduce it reliably.
I can verify that this is because of my PR. Minimal example that segfaults for me: And the reason for the segfault is apparent when run with simpler types This will print and continue in such a loop forever, printing the buffer over and over again. This happens because of line hitting an edge case when that causes head to be placed outside of the buffer. This can be fixed by changing that line to I think that is the only thing that is causing these segfaults.

} } <ins> #[test] fn issue_53529() { use boxed::Box; let mut dst = VecDeque::new(); dst.push_front(Box::new(1)); dst.push_front(Box::new(2)); assert_eq!(*dst.pop_back().unwrap(), 1); let mut src = VecDeque::new(); src.push_front(Box::new(2)); dst.append(&mut src); for a in dst { assert_eq!(*a, 2); } } </ins> }


Question:
here is the example: this example will compile well if you remove
Backtrace: `
Hmm I reproduced same ICE with next snippet without :
Smaller repro: Edit: AFAICT the problem is the existential assoc type is being made dependent on the type parameter of the trait method ('s in this repro, 's in the OP). So it would be a compile error if it weren't for the ICE.
compiles without ICE.
sorry it needs to be feature or used 2018 edition. Here is complete one which fails to compile regardless feature:
Right, 2015 doesn't hit the error until you actually try to use the type. Regardless, as I said in my previous comment, this code should be a compile error anyway regardless of edition, since it's trying to declare an associated type that depends on the type parameters of the trait method rather than the trait.
hey have you had a chance to look into this yet? Do you think you will be looking at it in the near term, or should we unassign you?
Removed my assignment. Unfortunately I'm running behind on the not-time-critical issues I assigned to myself.
Discussed in T-compiler meeting. This is a blocker for the existential type feature. But there is no schedule yet set for stabilizing that feature. Thus, downgrading fixing this to P-medium.
This no longer produces an ICE on the latest nightly.
Yes, it was fixed by

<ins> // ignore-tidy-linelength #![feature(type_alias_impl_trait)] use std::fmt::Debug; pub trait Foo { type Item: Debug; fn foo<T: Debug>(_: T) -> Self::Item; } #[derive(Debug)] pub struct S<T>(std::marker::PhantomData<T>); pub struct S2; impl Foo for S2 { type Item = impl Debug; fn foo<T: Debug>(_: T) -> Self::Item { //~^ Error type parameter `T` is part of concrete type but not used in parameter list for the `impl Trait` type alias S::<T>(Default::default()) } } fn main() { S2::foo(123); } </ins>


Question:
here is the example: this example will compile well if you remove
Backtrace: `
Hmm I reproduced same ICE with next snippet without :
Smaller repro: Edit: AFAICT the problem is the existential assoc type is being made dependent on the type parameter of the trait method ('s in this repro, 's in the OP). So it would be a compile error if it weren't for the ICE.
compiles without ICE.
sorry it needs to be feature or used 2018 edition. Here is complete one which fails to compile regardless feature:
Right, 2015 doesn't hit the error until you actually try to use the type. Regardless, as I said in my previous comment, this code should be a compile error anyway regardless of edition, since it's trying to declare an associated type that depends on the type parameters of the trait method rather than the trait.
hey have you had a chance to look into this yet? Do you think you will be looking at it in the near term, or should we unassign you?
Removed my assignment. Unfortunately I'm running behind on the not-time-critical issues I assigned to myself.
Discussed in T-compiler meeting. This is a blocker for the existential type feature. But there is no schedule yet set for stabilizing that feature. Thus, downgrading fixing this to P-medium.
This no longer produces an ICE on the latest nightly.
Yes, it was fixed by

<ins> error: type parameter `T` is part of concrete type but not used in parameter list for the `impl Trait` type alias --> $DIR/issue-53598.rs:20:42 | LL |       fn foo<T: Debug>(_: T) -> Self::Item { |  __________________________________________^ LL | | LL | |         S::<T>(Default::default()) LL | |     } | |_____^ error: aborting due to previous error </ins>


Question:
here is the example: this example will compile well if you remove
Backtrace: `
Hmm I reproduced same ICE with next snippet without :
Smaller repro: Edit: AFAICT the problem is the existential assoc type is being made dependent on the type parameter of the trait method ('s in this repro, 's in the OP). So it would be a compile error if it weren't for the ICE.
compiles without ICE.
sorry it needs to be feature or used 2018 edition. Here is complete one which fails to compile regardless feature:
Right, 2015 doesn't hit the error until you actually try to use the type. Regardless, as I said in my previous comment, this code should be a compile error anyway regardless of edition, since it's trying to declare an associated type that depends on the type parameters of the trait method rather than the trait.
hey have you had a chance to look into this yet? Do you think you will be looking at it in the near term, or should we unassign you?
Removed my assignment. Unfortunately I'm running behind on the not-time-critical issues I assigned to myself.
Discussed in T-compiler meeting. This is a blocker for the existential type feature. But there is no schedule yet set for stabilizing that feature. Thus, downgrading fixing this to P-medium.
This no longer produces an ICE on the latest nightly.
Yes, it was fixed by

<ins> // ignore-tidy-linelength #![feature(arbitrary_self_types)] #![feature(type_alias_impl_trait)] use std::ops::Deref; trait Foo { type Bar: Foo; fn foo(self: impl Deref<Target = Self>) -> Self::Bar; } impl<C> Foo for C { type Bar = impl Foo; fn foo(self: impl Deref<Target = Self>) -> Self::Bar { //~^ Error type parameter `impl Deref<Target = Self>` is part of concrete type but not used in parameter list for the `impl Trait` type alias self } } fn main() {} </ins>


Question:
here is the example: this example will compile well if you remove
Backtrace: `
Hmm I reproduced same ICE with next snippet without :
Smaller repro: Edit: AFAICT the problem is the existential assoc type is being made dependent on the type parameter of the trait method ('s in this repro, 's in the OP). So it would be a compile error if it weren't for the ICE.
compiles without ICE.
sorry it needs to be feature or used 2018 edition. Here is complete one which fails to compile regardless feature:
Right, 2015 doesn't hit the error until you actually try to use the type. Regardless, as I said in my previous comment, this code should be a compile error anyway regardless of edition, since it's trying to declare an associated type that depends on the type parameters of the trait method rather than the trait.
hey have you had a chance to look into this yet? Do you think you will be looking at it in the near term, or should we unassign you?
Removed my assignment. Unfortunately I'm running behind on the not-time-critical issues I assigned to myself.
Discussed in T-compiler meeting. This is a blocker for the existential type feature. But there is no schedule yet set for stabilizing that feature. Thus, downgrading fixing this to P-medium.
This no longer produces an ICE on the latest nightly.
Yes, it was fixed by

<ins> error: type parameter `impl Deref<Target = Self>` is part of concrete type but not used in parameter list for the `impl Trait` type alias --> $DIR/issue-57700.rs:16:58 | LL |       fn foo(self: impl Deref<Target = Self>) -> Self::Bar { |  __________________________________________________________^ LL | | LL | |         self LL | |     } | |_____^ error: aborting due to previous error </ins>


Question:
This takes a while to compile. I presume because needs to hit the loop checker in miri. Changing the foofoo&(loop{}, 1).1'staticlifetime. () Errors:
The following code times out with either borrow checker:
The reason that foo matters is because of
I don't think this is an NLL issue in particular?
Based on 's identification of a test that reproduces the issue on either borrow-checker, I am going to remove the A-NLL label.
That example is not problematic, because nothing is promoted at all. I now don't even think my example is problematic anymore, because it seems to be perfectly alright to promote anything if we diverge before we give the user access to the value. I guess this just needs a test now (just a ui test with a comment, not a run-pass test, as that would hang forever)
Can I work on this? I'm a newbie to rust.
Sure! You can take the example from the issue message and create a new file in and add the comment at the top. Then, when running it should succeed to run the tests.

<ins> //compile-pass #![feature(nll)] fn main() { let _: &'static usize = &(loop {}, 1).1; } </ins>


Question:
No matter what kind of const eval features we add, we should never allow this. Even if we managed to allow this, I think the current impl would ICE. Even if it does not ICE, it could never actually modify the memory. We should ensure that the following code always errors.
Hi , I would like to work on this if that's ok. I'm a new Rustacean and I'm learning rustc development and testing. It's time to get my feet wet
Great! I think the following steps should suffice, but don't hesitate to ask if anything is unclear or doesn't work as expected the test in a new file in a annotation to the line that should report an error (replace "error message here" with the relevant error message) 3 until everything is green your test and the generated file and open a PR to the PR message so when it gets merged this issue gets closed automatically
Hi , When I was running the test in step 3, or compiled the new test with nightly rustc (rust-lang/rust I got errors like this. error[E0015]: calls in statics are limited to constant functions, tuple structs and tuple variants --mod-static-with-const- | 24 | () = 5; | ^^^^^^^^^^^ error[E0658]: statements in statics are unstable (see issue ) --mod-static-with-const- | 24 | () = 5; | ^^^^^^^^^^^^^^^^ | = help: add #![feature(constlet)] to the crate attributes to enable error: aborting due to 2 previous errors I think E0015 is the expected error. But I'm not sure about E0658. It seems that the constlet feature is not stabilized yet. (But compiling doesn't return this error.) Could you please help with this problem? Thanks a lot.
These errors are fine. the idea of this issue is to just make sure the code always fails to compile. More errors are fine, too. the const let feature has a bug where even enabling it will still report it. that's not problematic for this issue. Can you add a comment to the test mentioning that this test should never ever succeed? If you want you can additionally try to minimize the test to something still emitting the feature gate error even though the feature gate is active (and maybe to something that should be perfectly fine at compile time). then open a new issue about that.
Thanks ! I will try to reproduce the unexpected error related to the feature gate. Meanwhile it seems that this task will be blocked by that issue since the ui test will fail if the error signature doesn't match with the expected one. Is that right?
you can just make them match ;) Add all the annotations you need to make the test pass. Whenever someone changes your test, because they fixed other bugs or implemented new features, they'll hopefully look at the comment about the fact that the test should never succeed to compile.

<ins> // Copyright 2018 The Rust Project Developers. See the COPYRIGHT // file at the top-level directory of this distribution and at // http://rust-lang.org/COPYRIGHT. // // Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or // http://www.apache.org/licenses/LICENSE-2.0> or the MIT license // <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your // option. This file may not be copied, modified, or distributed // except according to those terms. // New test for #53818: modifying static memory at compile-time is not allowed. // The test should never succeed. #![feature(const_raw_ptr_deref)] #![feature(const_let)] use std::cell::UnsafeCell; struct Foo(UnsafeCell<u32>); unsafe impl Send for Foo {} unsafe impl Sync for Foo {} static FOO: Foo = Foo(UnsafeCell::new(42)); static BAR: () = unsafe { *FOO.0.get() = 5; //~^ ERROR calls in statics are limited to constant functions, tuple structs and tuple variants // This error is caused by a separate bug that the feature gate error is reported // even though the feature gate "const_let" is active. //~| statements in statics are unstable (see issue #48821) }; fn main() { println!("{}", unsafe { *FOO.0.get() }); } </ins>


Question:
No matter what kind of const eval features we add, we should never allow this. Even if we managed to allow this, I think the current impl would ICE. Even if it does not ICE, it could never actually modify the memory. We should ensure that the following code always errors.
Hi , I would like to work on this if that's ok. I'm a new Rustacean and I'm learning rustc development and testing. It's time to get my feet wet
Great! I think the following steps should suffice, but don't hesitate to ask if anything is unclear or doesn't work as expected the test in a new file in a annotation to the line that should report an error (replace "error message here" with the relevant error message) 3 until everything is green your test and the generated file and open a PR to the PR message so when it gets merged this issue gets closed automatically
Hi , When I was running the test in step 3, or compiled the new test with nightly rustc (rust-lang/rust I got errors like this. error[E0015]: calls in statics are limited to constant functions, tuple structs and tuple variants --mod-static-with-const- | 24 | () = 5; | ^^^^^^^^^^^ error[E0658]: statements in statics are unstable (see issue ) --mod-static-with-const- | 24 | () = 5; | ^^^^^^^^^^^^^^^^ | = help: add #![feature(constlet)] to the crate attributes to enable error: aborting due to 2 previous errors I think E0015 is the expected error. But I'm not sure about E0658. It seems that the constlet feature is not stabilized yet. (But compiling doesn't return this error.) Could you please help with this problem? Thanks a lot.
These errors are fine. the idea of this issue is to just make sure the code always fails to compile. More errors are fine, too. the const let feature has a bug where even enabling it will still report it. that's not problematic for this issue. Can you add a comment to the test mentioning that this test should never ever succeed? If you want you can additionally try to minimize the test to something still emitting the feature gate error even though the feature gate is active (and maybe to something that should be perfectly fine at compile time). then open a new issue about that.
Thanks ! I will try to reproduce the unexpected error related to the feature gate. Meanwhile it seems that this task will be blocked by that issue since the ui test will fail if the error signature doesn't match with the expected one. Is that right?
you can just make them match ;) Add all the annotations you need to make the test pass. Whenever someone changes your test, because they fixed other bugs or implemented new features, they'll hopefully look at the comment about the fact that the test should never succeed to compile.

<ins> error[E0015]: calls in statics are limited to constant functions, tuple structs and tuple variants --> $DIR/mod-static-with-const-fn.rs:27:6 | LL |     *FOO.0.get() = 5; |      ^^^^^^^^^^^ error[E0658]: statements in statics are unstable (see issue #48821) --> $DIR/mod-static-with-const-fn.rs:27:5 | LL |     *FOO.0.get() = 5; |     ^^^^^^^^^^^^^^^^ | = help: add #![feature(const_let)] to the crate attributes to enable error: aborting due to 2 previous errors Some errors occurred: E0015, E0658. For more information about an error, try `rustc --explain E0015`. </ins>


Question: This also occurs with etc. Changing requires only the reverse dependencies of to be recompiled. is also recompiled. As you can see, this takes twice as much as compiling the reverse dependencies of alone, leading to compilation being 3x slower.

let mut cargo = Command::new(&self.initial_cargo); let out_dir = self.stage_out(compiler, mode); <del> // command specific path, we call clear_if_dirty with this let mut my_out = match cmd { "build" => self.cargo_out(compiler, mode, target), // This is the intended out directory for crate documentation. "doc" | "rustdoc" =>  self.crate_doc_out(target), _ => self.stage_out(compiler, mode), }; // This is for the original compiler, but if we're forced to use stage 1, then // std/test/rustc stamps won't exist in stage 2, so we need to get those from stage 1, since // we copy the libs forward. let cmp = self.compiler_for(compiler.stage, compiler.host, target); let libstd_stamp = match cmd { "check" | "clippy" | "fix" => check::libstd_stamp(self, cmp, target), _ => compile::libstd_stamp(self, cmp, target), }; let libtest_stamp = match cmd { "check" | "clippy" | "fix" => check::libtest_stamp(self, cmp, target), _ => compile::libtest_stamp(self, cmp, target), }; let librustc_stamp = match cmd { "check" | "clippy" | "fix" => check::librustc_stamp(self, cmp, target), _ => compile::librustc_stamp(self, cmp, target), }; </del> <ins> // Codegen backends are not yet tracked by -Zbinary-dep-depinfo, // so we need to explicitly clear out if they've been updated. for backend in self.codegen_backends(compiler) { self.clear_if_dirty(&out_dir, &backend); } </ins> if cmd == "doc" || cmd == "rustdoc" { <del> if mode == Mode::Rustc || mode == Mode::ToolRustc || mode == Mode::Codegen { </del> <ins> let my_out = match mode { </ins> // This is the intended out directory for compiler documentation. <del> my_out = self.compiler_doc_out(target); } </del> <ins> Mode::Rustc | Mode::ToolRustc | Mode::Codegen => self.compiler_doc_out(target), _ => self.crate_doc_out(target), }; </ins> let rustdoc = self.rustdoc(compiler); self.clear_if_dirty(&my_out, &rustdoc); <del> } else if cmd != "test" { match mode { Mode::Std => { self.clear_if_dirty(&my_out, &self.rustc(compiler)); for backend in self.codegen_backends(compiler) { self.clear_if_dirty(&my_out, &backend); } }, Mode::Test => { self.clear_if_dirty(&my_out, &libstd_stamp); }, Mode::Rustc => { self.clear_if_dirty(&my_out, &self.rustc(compiler)); self.clear_if_dirty(&my_out, &libstd_stamp); self.clear_if_dirty(&my_out, &libtest_stamp); }, Mode::Codegen => { self.clear_if_dirty(&my_out, &librustc_stamp); }, Mode::ToolBootstrap => { }, Mode::ToolStd => { self.clear_if_dirty(&my_out, &libstd_stamp); }, Mode::ToolTest => { self.clear_if_dirty(&my_out, &libstd_stamp); self.clear_if_dirty(&my_out, &libtest_stamp); }, Mode::ToolRustc => { self.clear_if_dirty(&my_out, &libstd_stamp); self.clear_if_dirty(&my_out, &libtest_stamp); self.clear_if_dirty(&my_out, &librustc_stamp); }, } </del> } cargo


Question: This also occurs with etc. Changing requires only the reverse dependencies of to be recompiled. is also recompiled. As you can see, this takes twice as much as compiling the reverse dependencies of alone, leading to compilation being 3x slower.

}, } <ins> // This tells Cargo (and in turn, rustc) to output more complete // dependency information.  Most importantly for rustbuild, this // includes sysroot artifacts, like libstd, which means that we don't // need to track those in rustbuild (an error prone process!). This // feature is currently unstable as there may be some bugs and such, but // it represents a big improvement in rustbuild's reliability on // rebuilds, so we're using it here. // // For some additional context, see #63470 (the PR originally adding // this), as well as #63012 which is the tracking issue for this // feature on the rustc side. cargo.arg("-Zbinary-dep-depinfo"); </ins> cargo.arg("-j").arg(self.jobs().to_string()); // Remove make-related flags to ensure Cargo can correctly set things up cargo.env_remove("MAKEFLAGS");


Question: This also occurs with etc. Changing requires only the reverse dependencies of to be recompiled. is also recompiled. As you can see, this takes twice as much as compiling the reverse dependencies of alone, leading to compilation being 3x slower.

let libdir = builder.sysroot_libdir(compiler, target); let hostdir = builder.sysroot_libdir(compiler, compiler.host); add_to_sysroot(&builder, &libdir, &hostdir, &rustdoc_stamp(builder, compiler, target)); <del> builder.cargo(compiler, Mode::ToolRustc, target, "clean"); </del> } }


Question: This also occurs with etc. Changing requires only the reverse dependencies of to be recompiled. is also recompiled. As you can see, this takes twice as much as compiling the reverse dependencies of alone, leading to compilation being 3x slower.

use std::process::{Command, Stdio, exit}; use std::str; <del> use build_helper::{output, mtime, t, up_to_date}; </del> <ins> use build_helper::{output, t, up_to_date}; </ins> use filetime::FileTime; use serde::Deserialize; use serde_json;


Question: This also occurs with etc. Changing requires only the reverse dependencies of to be recompiled. is also recompiled. As you can see, this takes twice as much as compiling the reverse dependencies of alone, leading to compilation being 3x slower.

// for reason why the sanitizers are not built in stage0. copy_apple_sanitizer_dylibs(builder, &builder.native_dir(target), "osx", &libdir); } <del> builder.cargo(target_compiler, Mode::ToolStd, target, "clean"); </del> } }


Question: This also occurs with etc. Changing requires only the reverse dependencies of to be recompiled. is also recompiled. As you can see, this takes twice as much as compiling the reverse dependencies of alone, leading to compilation being 3x slower.

&builder.sysroot_libdir(target_compiler, compiler.host), &libtest_stamp(builder, compiler, target) ); <del> builder.cargo(target_compiler, Mode::ToolTest, target, "clean"); </del> } }


Question: This also occurs with etc. Changing requires only the reverse dependencies of to be recompiled. is also recompiled. As you can see, this takes twice as much as compiling the reverse dependencies of alone, leading to compilation being 3x slower.

&builder.sysroot_libdir(target_compiler, compiler.host), &librustc_stamp(builder, compiler, target) ); <del> builder.cargo(target_compiler, Mode::ToolRustc, target, "clean"); </del> } }


Question: This also occurs with etc. Changing requires only the reverse dependencies of to be recompiled. is also recompiled. As you can see, this takes twice as much as compiling the reverse dependencies of alone, leading to compilation being 3x slower.

deps.push((path_to_add.into(), false)); } <del> // Now we want to update the contents of the stamp file, if necessary. First // we read off the previous contents along with its mtime. If our new // contents (the list of files to copy) is different or if any dep's mtime // is newer then we rewrite the stamp file. </del> deps.sort(); <del> let stamp_contents = fs::read(stamp); let stamp_mtime = mtime(&stamp); </del> let mut new_contents = Vec::new(); <del> let mut max = None; let mut max_path = None; </del> for (dep, proc_macro) in deps.iter() { <del> let mtime = mtime(dep); if Some(mtime) > max { max = Some(mtime); max_path = Some(dep.clone()); } </del> new_contents.extend(if *proc_macro { b"h" } else { b"t" }); new_contents.extend(dep.to_str().unwrap().as_bytes()); new_contents.extend(b"0"); } <del> let max = max.unwrap(); let max_path = max_path.unwrap(); let contents_equal = stamp_contents .map(|contents| contents == new_contents) .unwrap_or_default(); if contents_equal && max <= stamp_mtime { builder.verbose(&format!("not updating {:?}; contents equal and {:?} <= {:?}", stamp, max, stamp_mtime)); return deps.into_iter().map(|(d, _)| d).collect() } if max > stamp_mtime { builder.verbose(&format!("updating {:?} as {:?} changed", stamp, max_path)); } else { builder.verbose(&format!("updating {:?} as deps changed", stamp)); } </del> t!(fs::write(&stamp, &new_contents)); deps.into_iter().map(|(d, _)| d).collect() }


Question: Discovered in it turns out that our vendored copies of these libraires cause problems when they override the system versions by accident. We should rename our copies with a Rust-specific name to avoid this name clash.

fn copy_apple_sanitizer_dylibs(builder: &Builder, native_dir: &Path, platform: &str, into: &Path) { for &sanitizer in &["asan", "tsan"] { <del> let filename = format!("libclang_rt.{}_{}_dynamic.dylib", sanitizer, platform); </del> <ins> let filename = format!("lib__rustc__clang_rt.{}_{}_dynamic.dylib", sanitizer, platform); </ins> let mut src_path = native_dir.join(sanitizer); src_path.push("build"); src_path.push("lib");


Question: Discovered in it turns out that our vendored copies of these libraires cause problems when they override the system versions by accident. We should rename our copies with a Rust-specific name to avoid this name clash.

pub out_dir: PathBuf, } <ins> impl NativeLibBoilerplate { /// On OSX we don't want to ship the exact filename that compiler-rt builds. /// This conflicts with the system and ours is likely a wildly different /// version, so they can't be substituted. /// /// As a result, we rename it here but we need to also use /// `install_name_tool` on OSX to rename the commands listed inside of it to /// ensure it's linked against correctly. pub fn fixup_sanitizer_lib_name(&self, sanitizer_name: &str) { if env::var("TARGET").unwrap() != "x86_64-apple-darwin" { return } let dir = self.out_dir.join("build/lib/darwin"); let name = format!("clang_rt.{}_osx_dynamic", sanitizer_name); let src = dir.join(&format!("lib{}.dylib", name)); let new_name = format!("lib__rustc__{}.dylib", name); let dst = dir.join(&new_name); println!("{} => {}", src.display(), dst.display()); fs::rename(&src, &dst).unwrap(); let status = Command::new("install_name_tool") .arg("-id") .arg(format!("@rpath/{}", new_name)) .arg(&dst) .status() .expect("failed to execute `install_name_tool`"); assert!(status.success()); } } </ins> impl Drop for NativeLibBoilerplate { fn drop(&mut self) { if !thread::panicking() {


Question: Discovered in it turns out that our vendored copies of these libraires cause problems when they override the system versions by accident. We should rename our copies with a Rust-specific name to avoid this name clash.

pub fn sanitizer_lib_boilerplate(sanitizer_name: &str) -> Result<(NativeLibBoilerplate, String), ()> { <del> let (link_name, search_path, dynamic) = match &*env::var("TARGET").unwrap() { </del> <ins> let (link_name, search_path, apple) = match &*env::var("TARGET").unwrap() { </ins> "x86_64-unknown-linux-gnu" => ( format!("clang_rt.{}-x86_64", sanitizer_name), "build/lib/linux",


Question: Discovered in it turns out that our vendored copies of these libraires cause problems when they override the system versions by accident. We should rename our copies with a Rust-specific name to avoid this name clash.

), _ => return Err(()), }; <del> let to_link = if dynamic { format!("dylib={}", link_name) </del> <ins> let to_link = if apple { format!("dylib=__rustc__{}", link_name) </ins> } else { format!("static={}", link_name) };


Question: Discovered in it turns out that our vendored copies of these libraires cause problems when they override the system versions by accident. We should rename our copies with a Rust-specific name to avoid this name clash.

.out_dir(&native.out_dir) .build_target(&target) .build(); <ins> native.fixup_sanitizer_lib_name("asan"); </ins> } println!("cargo:rerun-if-env-changed=LLVM_CONFIG"); }


Question: Discovered in it turns out that our vendored copies of these libraires cause problems when they override the system versions by accident. We should rename our copies with a Rust-specific name to avoid this name clash.

.out_dir(&native.out_dir) .build_target(&target) .build(); <ins> native.fixup_sanitizer_lib_name("tsan"); </ins> } println!("cargo:rerun-if-env-changed=LLVM_CONFIG"); }


Question:
std::io::Seek I got confused by this line and over on so after a little discussion it seems like this might be better wording for those of us who are silly A seek beyond the end of a stream is allowed, but it is implementation-defined. The hyphen makes it clear “implementation defined” is a known concept (and matches it’s usage elsewhere) and the “it is” makes it a bit more clear that the full sentence is considered and intentional. While normally I find brevity improves clarity, in this case if you don’t realize “implementation defined” is a single concept, it sounds like a thought that trailed off after an edit
Thanks I know other projects I've worked with have Easy and Documentation as tags so I wasn't sure if you guys might do the same
Yep, they're just actual labels over on the right ;-) (I also tabbed away and forgot to put the easy label on immediately )
Don't worry about that! It's very easy for developers to misjudge whether such descriptions will make sense to the target users. Pointing these things out is a great way to contribute to documentation.

pub trait Seek { /// Seek to an offset, in bytes, in a stream. /// <del> /// A seek beyond the end of a stream is allowed, but implementation /// defined. </del> <ins> /// A seek beyond the end of a stream is allowed, but behavior is defined /// by the implementation. </ins> /// /// If the seek operation completed successfully, /// this method returns the new position from the start of the stream.


Question:
Compiling code like: results in: presumably because attempts to translate it into an x86-specific ABI, which the AArch64 backend rightly rejects. already ignores for non-x86 targets: I'm not completely sure what the right answer for Rust is. I think is the rough equivalent of the above clang bit and needs to take a few more cases into account for non-x86 Windows: does that sound right? If so, I'll code up a patch.
Sounds like a good place to me!

} impl Target { <del> /// Given a function ABI, turn "System" into the correct ABI for this target. </del> <ins> /// Given a function ABI, turn it into the correct ABI for this target. </ins> pub fn adjust_abi(&self, abi: Abi) -> Abi { match abi { Abi::System => {


Question:
Compiling code like: results in: presumably because attempts to translate it into an x86-specific ABI, which the AArch64 backend rightly rejects. already ignores for non-x86 targets: I'm not completely sure what the right answer for Rust is. I think is the rough equivalent of the above clang bit and needs to take a few more cases into account for non-x86 Windows: does that sound right? If so, I'll code up a patch.
Sounds like a good place to me!

Abi::C } }, <ins> // These ABI kinds are ignored on non-x86 Windows targets. // See https://docs.microsoft.com/en-us/cpp/cpp/argument-passing-and-naming-conventions // and the individual pages for __stdcall et al. Abi::Stdcall | Abi::Fastcall | Abi::Vectorcall | Abi::Thiscall => { if self.options.is_like_windows && self.arch != "x86" { Abi::C } else { abi } }, </ins> abi => abi } }


Question:
Compiling code like: results in: presumably because attempts to translate it into an x86-specific ABI, which the AArch64 backend rightly rejects. already ignores for non-x86 targets: I'm not completely sure what the right answer for Rust is. I think is the rough equivalent of the above clang bit and needs to take a few more cases into account for non-x86 Windows: does that sound right? If so, I'll code up a patch.
Sounds like a good place to me!

// option. This file may not be copied, modified, or distributed // except according to those terms. <del> // ignore-arm // ignore-aarch64 </del> <ins> // Test that `extern "stdcall"` is properly translated. // only-x86 </ins> // compile-flags: -C no-prepopulate-passes


Question:
Because LLVM doesn't know what is, we currently , relying on the  function. So if the current CPU's family is Haswell, we convert to . As noticed and , some Intel Pentiums belong to the Haswell microarch, but they lack AVX. <hr/Here are the flags Clang generates for : Not only does it set the right for optimization purposes, it actually manually enables / disables all the features available / not available on the host. This ensures no matter what the microarch is, only available features will be used. <hr/We should probably use the . I think this issue will supersede both rust-lang/rust and rust-lang/rust. If we can fix this, we will never generate code with unavailable target features. Reproducing these errors is pretty hard, unless you have a machine with a Haswell Intel Pentium. I think it might be possible to reproduce this in QEMU with
Alternatively, we could also use the macros to do this. I'll work on this.
To add another example, doesn't detect the feature on broadwell (see ).
I submitted a PR for a

let code_model = to_llvm_code_model(sess.code_model()); <del> let features = attributes::llvm_target_features(sess).collect::<Vec<_>>(); </del> <ins> let mut features = llvm_util::handle_native_features(sess); features.extend(attributes::llvm_target_features(sess).map(|s| s.to_owned())); </ins> let mut singlethread = sess.target.singlethread; // On the wasm target once the `atomics` feature is enabled that means that


Question:
Because LLVM doesn't know what is, we currently , relying on the  function. So if the current CPU's family is Haswell, we convert to . As noticed and , some Intel Pentiums belong to the Haswell microarch, but they lack AVX. <hr/Here are the flags Clang generates for : Not only does it set the right for optimization purposes, it actually manually enables / disables all the features available / not available on the host. This ensures no matter what the microarch is, only available features will be used. <hr/We should probably use the . I think this issue will supersede both rust-lang/rust and rust-lang/rust. If we can fix this, we will never generate code with unavailable target features. Reproducing these errors is pretty hard, unless you have a machine with a Haswell Intel Pentium. I think it might be possible to reproduce this in QEMU with
Alternatively, we could also use the macros to do this. I'll work on this.
To add another example, doesn't detect the feature on broadwell (see ).
I submitted a PR for a

PM: &PassManager<'_>, ); <ins> pub fn LLVMGetHostCPUFeatures() -> *mut c_char; pub fn LLVMDisposeMessage(message: *mut c_char); </ins> // Stuff that's in llvm-wrapper/ because it's not upstream yet. /// Opens an object file.


Question:
Because LLVM doesn't know what is, we currently , relying on the  function. So if the current CPU's family is Haswell, we convert to . As noticed and , some Intel Pentiums belong to the Haswell microarch, but they lack AVX. <hr/Here are the flags Clang generates for : Not only does it set the right for optimization purposes, it actually manually enables / disables all the features available / not available on the host. This ensures no matter what the microarch is, only available features will be used. <hr/We should probably use the . I think this issue will supersede both rust-lang/rust and rust-lang/rust. If we can fix this, we will never generate code with unavailable target features. Reproducing these errors is pretty hard, unless you have a machine with a Haswell Intel Pentium. I think it might be possible to reproduce this in QEMU with
Alternatively, we could also use the macros to do this. I'll work on this.
To add another example, doesn't detect the feature on broadwell (see ).
I submitted a PR for a

use rustc_session::Session; use rustc_span::symbol::Symbol; use rustc_target::spec::{MergeFunctions, PanicStrategy}; <del> use std::ffi::CString; </del> <ins> use std::ffi::{CStr, CString}; </ins> use std::slice; use std::str;


Question:
Because LLVM doesn't know what is, we currently , relying on the  function. So if the current CPU's family is Haswell, we convert to . As noticed and , some Intel Pentiums belong to the Haswell microarch, but they lack AVX. <hr/Here are the flags Clang generates for : Not only does it set the right for optimization purposes, it actually manually enables / disables all the features available / not available on the host. This ensures no matter what the microarch is, only available features will be used. <hr/We should probably use the . I think this issue will supersede both rust-lang/rust and rust-lang/rust. If we can fix this, we will never generate code with unavailable target features. Reproducing these errors is pretty hard, unless you have a machine with a Haswell Intel Pentium. I think it might be possible to reproduce this in QEMU with
Alternatively, we could also use the macros to do this. I'll work on this.
To add another example, doesn't detect the feature on broadwell (see ).
I submitted a PR for a

handle_native(name) } <ins> pub fn handle_native_features(sess: &Session) -> Vec<String> { match sess.opts.cg.target_cpu { Some(ref s) => { if s != "native" { return vec![]; } let features_string = unsafe { let ptr = llvm::LLVMGetHostCPUFeatures(); let features_string = if !ptr.is_null() { CStr::from_ptr(ptr) .to_str() .unwrap_or_else(|e| { bug!("LLVM returned a non-utf8 features string: {}", e); }) .to_owned() } else { bug!("could not allocate host CPU features, LLVM returned a `null` string"); }; llvm::LLVMDisposeMessage(ptr); features_string }; features_string.split(",").map(|s| s.to_owned()).collect() } None => vec![], } } </ins> pub fn tune_cpu(sess: &Session) -> Option<&str> { match sess.opts.debugging_opts.tune_cpu { Some(ref s) => Some(handle_native(&**s)),


Question:
Rust version: How to reproduce: Error:
Minimized reproduction:
I'd like to work on this.

ty::RegionKind::ReLateBound(_, _), ) => {} <del> (ty::RegionKind::ReLateBound(_, _), _) => { </del> <ins> (ty::RegionKind::ReLateBound(_, _), _) | (_, ty::RegionKind::ReVar(_)) => { // One of these is true: </ins> // The new predicate has a HRTB in a spot where the old // predicate does not (if they both had a HRTB, the previous <del> // match arm would have executed). </del> <ins> // match arm would have executed). A HRBT is a 'stricter' // bound than anything else, so we want to keep the newer // predicate (with the HRBT) in place of the old predicate. </ins> // <del> // The means we want to remove the older predicate from // user_computed_preds, since having both it and the new </del> <ins> // OR // // The old predicate has a region variable where the new // predicate has some other kind of region. An region // variable isn't something we can actually display to a user, // so we choose ther new predicate (which doesn't have a region // varaible). // // In both cases, we want to remove the old predicate, // from user_computed_preds, and replace it with the new // one. Having both the old and the new </ins> // predicate in a ParamEnv would confuse SelectionContext <ins> // </ins> // We're currently in the predicate passed to 'retain', // so we return 'false' to remove the old predicate from // user_computed_preds return false; } <del> (_, ty::RegionKind::ReLateBound(_, _)) => { // This is the opposite situation as the previous arm - the // old predicate has a HRTB lifetime in a place where the // new predicate does not. We want to leave the old </del> <ins> (_, ty::RegionKind::ReLateBound(_, _)) | (ty::RegionKind::ReVar(_), _) => { // This is the opposite situation as the previous arm. // One of these is true: // // The old predicate has a HRTB lifetime in a place where the // new predicate does not. // // OR // // The new predicate has a region variable where the old // predicate has some other type of region. // // We want to leave the old </ins> // predicate in user_computed_preds, and skip adding // new_pred to user_computed_params. should_add_new = false <del> } </del> <ins> }, </ins> _ => {} } }