Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dependency errors #5319

Closed
p6rt opened this issue May 16, 2016 · 26 comments
Closed

dependency errors #5319

p6rt opened this issue May 16, 2016 · 26 comments

Comments

@p6rt
Copy link

p6rt commented May 16, 2016

Migrated from rt.perl.org#128156 (status was 'resolved')

Searchable as RT128156$

@p6rt
Copy link
Author

p6rt commented May 16, 2016

From @MARTIMM

Hi,

I get the following error more often than before after some edits in my
source files.

Missing or wrong version of dependency
'/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Wire.pm6
(MongoDB​::Wire)' (from
'/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Cursor.pm6
(MongoDB​::Cursor)'

A remedy to this is to touch all source files in the library I work on
and than retry. I think that this should not happen, at least not that
often (I've seen it several times in about a week I think, twice this
evening, to give an idea).

This is Rakudo version 2016.04-200-gad82657 built on MoarVM version
2016.04-134-g9879233
implementing Perl 6.c.

Greetings,
Marcel

@p6rt
Copy link
Author

p6rt commented May 20, 2016

From @LLFourn

I've been getting something similar and I've managed to golf it.

mkdir lib;
echo 'class One { }; need Two;' > lib/One.pm6
echo 'class Two { }; need Three;' > lib/Two.pm6
echo '' > lib/Three.pm6
perl6 -Ilib -e 'need One; say (GLOBAL​::<One>​:exists ?? "OK" !! "NOT OK")'
echo "changing file" && touch lib/Two.pm6 # Three.pm6 will also break
perl6 -Ilib -e 'need One; say (GLOBAL​::<One>​:exists ?? "OK" !! "NOT OK")'

summary​: Precomp is broken for dependency chains 3 or longer when one of
the dependencies down the chain is broken.

Do we have a way of writing tests for precomp?

On Tue, May 17, 2016 at 4​:23 AM mt1957 <perl6-bugs-followup@​perl.org> wrote​:

# New Ticket Created by mt1957
# Please include the string​: [perl #​128156]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl6/Ticket/Display.html?id=128156 >

Hi,

I get the following error more often than before after some edits in my
source files.

Missing or wrong version of dependency

'/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Wire.pm6
(MongoDB​::Wire)' (from

'/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Cursor.pm6
(MongoDB​::Cursor)'

A remedy to this is to touch all source files in the library I work on
and than retry. I think that this should not happen, at least not that
often (I've seen it several times in about a week I think, twice this
evening, to give an idea).

This is Rakudo version 2016.04-200-gad82657 built on MoarVM version
2016.04-134-g9879233
implementing Perl 6.c.

Greetings,
Marcel

@p6rt
Copy link
Author

p6rt commented May 20, 2016

The RT System itself - Status changed from 'new' to 'open'

@p6rt
Copy link
Author

p6rt commented May 20, 2016

From @LLFourn

err leme try that again

Precomp is broken for dependency chains 3 or longer when one of the
dependencies down the chain is *changed*.

To expand​: Sometimes it gives you a problem where you get "Missing or wrong
version of dependency" and sometimes symbols that should have been merged
from GLOBALish just aren't there which is the case in the example above.

On Fri, May 20, 2016 at 11​:18 PM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

I've been getting something similar and I've managed to golf it.

mkdir lib;
echo 'class One { }; need Two;' > lib/One.pm6
echo 'class Two { }; need Three;' > lib/Two.pm6
echo '' > lib/Three.pm6
perl6 -Ilib -e 'need One; say (GLOBAL​::<One>​:exists ?? "OK" !! "NOT OK")'
echo "changing file" && touch lib/Two.pm6 # Three.pm6 will also break
perl6 -Ilib -e 'need One; say (GLOBAL​::<One>​:exists ?? "OK" !! "NOT OK")'

summary​: Precomp is broken for dependency chains 3 or longer when one of
the dependencies down the chain is broken.

Do we have a way of writing tests for precomp?

On Tue, May 17, 2016 at 4​:23 AM mt1957 <perl6-bugs-followup@​perl.org>
wrote​:

# New Ticket Created by mt1957
# Please include the string​: [perl #​128156]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl6/Ticket/Display.html?id=128156 >

Hi,

I get the following error more often than before after some edits in my
source files.

Missing or wrong version of dependency

'/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Wire.pm6
(MongoDB​::Wire)' (from

'/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Cursor.pm6
(MongoDB​::Cursor)'

A remedy to this is to touch all source files in the library I work on
and than retry. I think that this should not happen, at least not that
often (I've seen it several times in about a week I think, twice this
evening, to give an idea).

This is Rakudo version 2016.04-200-gad82657 built on MoarVM version
2016.04-134-g9879233
implementing Perl 6.c.

Greetings,
Marcel

@p6rt
Copy link
Author

p6rt commented May 20, 2016

From @LLFourn

It seems to have been introduced by one of nine's commits on the 13th. This
or one of the commits just before it I think​:

rakudo/rakudo@4fb3f94

On Fri, May 20, 2016 at 11​:26 PM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

err leme try that again

Precomp is broken for dependency chains 3 or longer when one of the
dependencies down the chain is *changed*.

To expand​: Sometimes it gives you a problem where you get "Missing or
wrong version of dependency" and sometimes symbols that should have been
merged from GLOBALish just aren't there which is the case in the example
above.

On Fri, May 20, 2016 at 11​:18 PM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

I've been getting something similar and I've managed to golf it.

mkdir lib;
echo 'class One { }; need Two;' > lib/One.pm6
echo 'class Two { }; need Three;' > lib/Two.pm6
echo '' > lib/Three.pm6
perl6 -Ilib -e 'need One; say (GLOBAL​::<One>​:exists ?? "OK" !! "NOT OK")'
echo "changing file" && touch lib/Two.pm6 # Three.pm6 will also break
perl6 -Ilib -e 'need One; say (GLOBAL​::<One>​:exists ?? "OK" !! "NOT OK")'

summary​: Precomp is broken for dependency chains 3 or longer when one
of the dependencies down the chain is broken.

Do we have a way of writing tests for precomp?

On Tue, May 17, 2016 at 4​:23 AM mt1957 <perl6-bugs-followup@​perl.org>
wrote​:

# New Ticket Created by mt1957
# Please include the string​: [perl #​128156]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl6/Ticket/Display.html?id=128156 >

Hi,

I get the following error more often than before after some edits in my
source files.

Missing or wrong version of dependency

'/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Wire.pm6
(MongoDB​::Wire)' (from

'/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Cursor.pm6
(MongoDB​::Cursor)'

A remedy to this is to touch all source files in the library I work on
and than retry. I think that this should not happen, at least not that
often (I've seen it several times in about a week I think, twice this
evening, to give an idea).

This is Rakudo version 2016.04-200-gad82657 built on MoarVM version
2016.04-134-g9879233
implementing Perl 6.c.

Greetings,
Marcel

@p6rt
Copy link
Author

p6rt commented May 20, 2016

From @niner

Committed a fix in c59e4dc44772cb09edeb8aa7f0ce0385f951cf5d

Would be extremely helpful if you could translate your excellent test case into a proper spec test :)

@p6rt
Copy link
Author

p6rt commented May 21, 2016

From @LLFourn

nine++ has attempted to fix this issue​:

rakudo/rakudo@c59e4dc

Though it seems to have made the failure more random.

Try the following​:
while true; do
  test -e lib && rm -r lib; # this is needed to trigger it for some reason
  mkdir lib;
  echo 'class One { }; need Two;' > lib/One.pm6
  echo 'class Two { }; need Three;' > lib/Two.pm6
  echo '' > lib/Three.pm6
  perl6 -Ilib -e 'need One;'
  touch lib/Three.pm6
  perl6 -Ilib -e 'need One; say GLOBAL​::.keys;'
done

gives me​:

(Two One)
()
(Two One)
()
(Two One)
()
(Two One)
(Two One)
(Two One)
(Two One)
(Two One)
()

I wrote a test and push it to roast.

Raku/roast@4e8b2e0

It seems to have the same issue.

On Sat, May 21, 2016 at 1​:45 AM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

It seems to have been introduced by one of nine's commits on the 13th.
This or one of the commits just before it I think​:

rakudo/rakudo@4fb3f94

On Fri, May 20, 2016 at 11​:26 PM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

err leme try that again

Precomp is broken for dependency chains 3 or longer when one of the
dependencies down the chain is *changed*.

To expand​: Sometimes it gives you a problem where you get "Missing or
wrong version of dependency" and sometimes symbols that should have been
merged from GLOBALish just aren't there which is the case in the example
above.

On Fri, May 20, 2016 at 11​:18 PM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

I've been getting something similar and I've managed to golf it.

mkdir lib;
echo 'class One { }; need Two;' > lib/One.pm6
echo 'class Two { }; need Three;' > lib/Two.pm6
echo '' > lib/Three.pm6
perl6 -Ilib -e 'need One; say (GLOBAL​::<One>​:exists ?? "OK" !! "NOT OK")'
echo "changing file" && touch lib/Two.pm6 # Three.pm6 will also break
perl6 -Ilib -e 'need One; say (GLOBAL​::<One>​:exists ?? "OK" !! "NOT OK")'

summary​: Precomp is broken for dependency chains 3 or longer when one
of the dependencies down the chain is broken.

Do we have a way of writing tests for precomp?

On Tue, May 17, 2016 at 4​:23 AM mt1957 <perl6-bugs-followup@​perl.org>
wrote​:

# New Ticket Created by mt1957
# Please include the string​: [perl #​128156]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl6/Ticket/Display.html?id=128156 >

Hi,

I get the following error more often than before after some edits in my
source files.

Missing or wrong version of dependency

'/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Wire.pm6
(MongoDB​::Wire)' (from

'/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Cursor.pm6
(MongoDB​::Cursor)'

A remedy to this is to touch all source files in the library I work on
and than retry. I think that this should not happen, at least not that
often (I've seen it several times in about a week I think, twice this
evening, to give an idea).

This is Rakudo version 2016.04-200-gad82657 built on MoarVM version
2016.04-134-g9879233
implementing Perl 6.c.

Greetings,
Marcel

@p6rt
Copy link
Author

p6rt commented May 22, 2016

From @niner

Sorry, I cannot reproduce the new failure here. precompilation.rakudo.moar passed 150 consecutive times and your shell script printed (Two One) 500 times in a row before I stopped them.

@p6rt
Copy link
Author

p6rt commented May 23, 2016

From @LLFourn

I've done some more investigation. There are two seperate issues​:

1. The main issue wrt this ticket. It hits you will you are editing code.
It can be reproduced (on any platform) with​:

P6="perl6"
lib="templib"
test -e $lib && rm -r $lib;
mkdir $lib
sleep="sleep 2";
echo 'class Top1 { }; need Needed;' > $lib/Top1.pm6
echo 'class Needed { };' > $lib/Needed.pm6;
echo 'class Top2 { }; need Needed;' > $lib/Top2.pm6;

while true; do
  printf '#' >> $lib/Needed.pm6 # touch isn't enough here
  $sleep;
  $P6 -I$lib -e 'need Top1; need Top2; say GLOBAL​::.keys;'
done

====
which will fail on the second iteration of the loop​:

(Top2 Needed Top1)
===SORRY!===
Missing or wrong version of dependency '/root/templib3/Needed.pm6 (Needed)'
(from '/root/templib3/Top2.pm6 (Top2)')

This one is different because the content of the file has to actually
change rather than just the mtime to trigger it.

2. A smaller issue which is what I was investigating in the prior response.
If you load/recompile a compunit in the same second as the source is
modified it will fail. (your processor and disk has to be fast enough to
pull this off which I *think* is why nine couldn't reproduce it rather than
this being a mac specific thing).

This race condition essentially happens in this line in !load-dependencies​:

return False if not $srcIO.e or $modified <= $srcIO.modified;

because it's <= it opens the possibility that a precomp is marked as out of
date a second time even after it has just been re-precompiled in the same
process. The consequence seems to be a bad serialisation. (of course I'm
not suggesting it shouldn't be <= but I guess the fact that something has
just been re-precompiled should override this check).

On Sun, May 22, 2016 at 7​:12 AM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

nine++ has attempted to fix this issue​:

rakudo/rakudo@c59e4dc

Though it seems to have made the failure more random.

Try the following​:
while true; do
test -e lib && rm -r lib; # this is needed to trigger it for some
reason
mkdir lib;
echo 'class One { }; need Two;' > lib/One.pm6
echo 'class Two { }; need Three;' > lib/Two.pm6
echo '' > lib/Three.pm6
perl6 -Ilib -e 'need One;'
touch lib/Three.pm6
perl6 -Ilib -e 'need One; say GLOBAL​::.keys;'
done

gives me​:

(Two One)
()
(Two One)
()
(Two One)
()
(Two One)
(Two One)
(Two One)
(Two One)
(Two One)
()

I wrote a test and push it to roast.

Raku/roast@4e8b2e0

It seems to have the same issue.

On Sat, May 21, 2016 at 1​:45 AM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

It seems to have been introduced by one of nine's commits on the 13th.
This or one of the commits just before it I think​:

rakudo/rakudo@4fb3f94

On Fri, May 20, 2016 at 11​:26 PM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

err leme try that again

Precomp is broken for dependency chains 3 or longer when one of the
dependencies down the chain is *changed*.

To expand​: Sometimes it gives you a problem where you get "Missing or
wrong version of dependency" and sometimes symbols that should have been
merged from GLOBALish just aren't there which is the case in the example
above.

On Fri, May 20, 2016 at 11​:18 PM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

I've been getting something similar and I've managed to golf it.

mkdir lib;
echo 'class One { }; need Two;' > lib/One.pm6
echo 'class Two { }; need Three;' > lib/Two.pm6
echo '' > lib/Three.pm6
perl6 -Ilib -e 'need One; say (GLOBAL​::<One>​:exists ?? "OK" !! "NOT
OK")'
echo "changing file" && touch lib/Two.pm6 # Three.pm6 will also break
perl6 -Ilib -e 'need One; say (GLOBAL​::<One>​:exists ?? "OK" !! "NOT
OK")'

summary​: Precomp is broken for dependency chains 3 or longer when one
of the dependencies down the chain is broken.

Do we have a way of writing tests for precomp?

On Tue, May 17, 2016 at 4​:23 AM mt1957 <perl6-bugs-followup@​perl.org>
wrote​:

# New Ticket Created by mt1957
# Please include the string​: [perl #​128156]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl6/Ticket/Display.html?id=128156 >

Hi,

I get the following error more often than before after some edits in my
source files.

Missing or wrong version of dependency

'/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Wire.pm6
(MongoDB​::Wire)' (from

'/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Cursor.pm6
(MongoDB​::Cursor)'

A remedy to this is to touch all source files in the library I work on
and than retry. I think that this should not happen, at least not that
often (I've seen it several times in about a week I think, twice this
evening, to give an idea).

This is Rakudo version 2016.04-200-gad82657 built on MoarVM version
2016.04-134-g9879233
implementing Perl 6.c.

Greetings,
Marcel

@p6rt
Copy link
Author

p6rt commented May 23, 2016

From @niner

On Sun May 22 17​:14​:16 2016, lloyd.fourn@​gmail.com wrote​:

This one is different because the content of the file has to actually
change rather than just the mtime to trigger it.

This sounds worrisome. I have no idea why that should make a difference. Will try to debug it anyway.

because it's <= it opens the possibility that a precomp is marked as
out of
date a second time even after it has just been re-precompiled in the
same
process. The consequence seems to be a bad serialisation. (of course

The consequence should be that we just needlessly recompile the file, even though it has just been compiled successfully, i.e. a bit of harmless waste of time. If we compared with < it could happen that the source was modified in the same second as the old version was precompiled and we miss the update. Will have to think those interactions through one more time.

@p6rt
Copy link
Author

p6rt commented May 23, 2016

From @LLFourn

wrt to outdating files. What I would actually expect from looking at the
code is for it to fail to produce a valid $handle and fallback to
non-precomp loading.

  my $handle = (
  self.may-precomp and (
  my $loaded = self.load($id, :since($source.modified),
:@​precomp-stores) # already precompiled?
  or self.precompile($source, $id,
:source-name($dependency.source-name), :force($loaded ~~ Failure))
  and self.load($id, :@​precomp-stores) # if not do it now
  )
  );

1. File changes at timestamp X
2. p6 is run at timestamp X
3. p6 detects file has changed in the first self.load() (so it returns a
fail "Outdated precompiled $unit")
4. so it does a :force recompile
5. then tries to self.load() a second time -- but because we are still at
time X the second load still returns failure
6. $precompiled gets set to False (which means a fallback non-precomp load
should be used)

After a few "note"s around the place the above seems to be what is
happening but I could be wrong.
I'm not sure how we end up with the current failure mode (symbols missing
from GLOBAL). The only thing that stands out to me is that it's
nqp​::exit(0) ie exit success, when precomp fails. I don't have a picture in
my mind of the whole process yet so exit(0) might make sense. This
precompilation is complicated stuff!

On Mon, May 23, 2016 at 10​:13 AM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

I've done some more investigation. There are two seperate issues​:

1. The main issue wrt this ticket. It hits you will you are editing code.
It can be reproduced (on any platform) with​:

P6="perl6"
lib="templib"
test -e $lib && rm -r $lib;
mkdir $lib
sleep="sleep 2";
echo 'class Top1 { }; need Needed;' > $lib/Top1.pm6
echo 'class Needed { };' > $lib/Needed.pm6;
echo 'class Top2 { }; need Needed;' > $lib/Top2.pm6;

while true; do
printf '#' >> $lib/Needed.pm6 # touch isn't enough here
$sleep;
$P6 -I$lib -e 'need Top1; need Top2; say GLOBAL​::.keys;'
done

====
which will fail on the second iteration of the loop​:

(Top2 Needed Top1)
===SORRY!===
Missing or wrong version of dependency '/root/templib3/Needed.pm6
(Needed)' (from '/root/templib3/Top2.pm6 (Top2)')

This one is different because the content of the file has to actually
change rather than just the mtime to trigger it.

2. A smaller issue which is what I was investigating in the prior
response. If you load/recompile a compunit in the same second as the source
is modified it will fail. (your processor and disk has to be fast enough to
pull this off which I *think* is why nine couldn't reproduce it rather than
this being a mac specific thing).

This race condition essentially happens in this line in
!load-dependencies​:

return False if not $srcIO.e or $modified <= $srcIO.modified;

because it's <= it opens the possibility that a precomp is marked as out
of date a second time even after it has just been re-precompiled in the
same process. The consequence seems to be a bad serialisation. (of course
I'm not suggesting it shouldn't be <= but I guess the fact that something
has just been re-precompiled should override this check).

On Sun, May 22, 2016 at 7​:12 AM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

nine++ has attempted to fix this issue​:

rakudo/rakudo@c59e4dc

Though it seems to have made the failure more random.

Try the following​:
while true; do
test -e lib && rm -r lib; # this is needed to trigger it for some
reason
mkdir lib;
echo 'class One { }; need Two;' > lib/One.pm6
echo 'class Two { }; need Three;' > lib/Two.pm6
echo '' > lib/Three.pm6
perl6 -Ilib -e 'need One;'
touch lib/Three.pm6
perl6 -Ilib -e 'need One; say GLOBAL​::.keys;'
done

gives me​:

(Two One)
()
(Two One)
()
(Two One)
()
(Two One)
(Two One)
(Two One)
(Two One)
(Two One)
()

I wrote a test and push it to roast.

Raku/roast@4e8b2e0

It seems to have the same issue.

On Sat, May 21, 2016 at 1​:45 AM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

It seems to have been introduced by one of nine's commits on the 13th.
This or one of the commits just before it I think​:

rakudo/rakudo@4fb3f94

On Fri, May 20, 2016 at 11​:26 PM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

err leme try that again

Precomp is broken for dependency chains 3 or longer when one of the
dependencies down the chain is *changed*.

To expand​: Sometimes it gives you a problem where you get "Missing or
wrong version of dependency" and sometimes symbols that should have been
merged from GLOBALish just aren't there which is the case in the example
above.

On Fri, May 20, 2016 at 11​:18 PM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

I've been getting something similar and I've managed to golf it.

mkdir lib;
echo 'class One { }; need Two;' > lib/One.pm6
echo 'class Two { }; need Three;' > lib/Two.pm6
echo '' > lib/Three.pm6
perl6 -Ilib -e 'need One; say (GLOBAL​::<One>​:exists ?? "OK" !! "NOT
OK")'
echo "changing file" && touch lib/Two.pm6 # Three.pm6 will also break
perl6 -Ilib -e 'need One; say (GLOBAL​::<One>​:exists ?? "OK" !! "NOT
OK")'

summary​: Precomp is broken for dependency chains 3 or longer when
one of the dependencies down the chain is broken.

Do we have a way of writing tests for precomp?

On Tue, May 17, 2016 at 4​:23 AM mt1957 <perl6-bugs-followup@​perl.org>
wrote​:

# New Ticket Created by mt1957
# Please include the string​: [perl #​128156]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl6/Ticket/Display.html?id=128156 >

Hi,

I get the following error more often than before after some edits in
my
source files.

Missing or wrong version of dependency

'/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Wire.pm6
(MongoDB​::Wire)' (from

'/home/marcel/Languages/Perl6/Projects/mongo-perl6-driver/lib/MongoDB/Cursor.pm6
(MongoDB​::Cursor)'

A remedy to this is to touch all source files in the library I work on
and than retry. I think that this should not happen, at least not that
often (I've seen it several times in about a week I think, twice this
evening, to give an idea).

This is Rakudo version 2016.04-200-gad82657 built on MoarVM version
2016.04-134-g9879233
implementing Perl 6.c.

Greetings,
Marcel

@p6rt
Copy link
Author

p6rt commented May 24, 2016

From @MARTIMM

Hi,

Another symptom from the same bug. After modifying a sub module I get an
error like the following;

Type 'Semi-xml​::Sxml' is not declared
at /home/marcel/Languages/Perl6/Projects/Semi-xml/t/108-xml.t​:43
------> my Semi-xml​::Sxml⏏ $x .= new;
Malformed my
at /home/marcel/Languages/Perl6/Projects/Semi-xml/t/108-xml.t​:43
------> my Semi-xml​::⏏Sxml $x .= new;

Then, after modifying the top module (just a simple say), everything
runs ok again.

This is Rakudo version 2016.04-209-gdb10a01 built on MoarVM version
2016.04-134-g9879233

Greetings,
Marcel

@p6rt
Copy link
Author

p6rt commented May 27, 2016

From @niner

Just a quick heads up​: I've managed to reproduce the errors with the second test.sh script by running it on a vfat file system. That increases my suspicion that it's caused by coarse time stamps. At least one issue, as there could be several.

@p6rt
Copy link
Author

p6rt commented May 29, 2016

From @niner

I fear I have to rewrite the whole dependency loading code. Right now we try to load a precomp file, during loading of the dependencies notice that one of them changed, precompile the source again and load the result. Sometimes compilation is so fast that it finishes in the same Instant (with just 2 second resolution on crappy file systems) as the dependency's. Then we declare it outdated though we just precompiled it and try to load from source.

Problem is that we really don't know which of them is older when the time stamps match, so we cannot rely on the target module being fresher even though we just compiled it ourselves. Someone else could have recompiled the dependency in the meantime.

All of this was much easier when we locked the whole precomp store during module loading. But a global lock is a luxury we just cannot afford :) The solution will probably be to first check all dependencies, then open all dependencies' files, then check again and only if they still check out actually pass them to the CompUnit​::Loader. If they don't check out again, rinse and repeat.

@p6rt
Copy link
Author

p6rt commented May 30, 2016

From @niner

Could you please try the rt128156_fix_precomp_deps_validation branch? It contains a possible fix.

@p6rt
Copy link
Author

p6rt commented Jun 6, 2016

From @FROGGS

I've got the following files​:

~/dev/rakudo/test$ tree .
.
├── A.pm
├── B.pm
└── C.pm

$ cat A.pm
use C :foo;
use B;

$ cat B.pm
use C :foo;

$ cat C.pm
use v6;
use nqp;
use NativeCall;
my class CStruct is repr('CStruct') is export(​:foo) { }
# my class CStruct is repr('CStruct') { }

When I run this code once​:
$ perl6 -I. -MA -e ''

All is fine... But when I run it again after adding an empty line to
file C.pm I get​:
  perl6 -I. -MA -e ''
===SORRY!===
Missing or wrong version of dependency
'/home/froggs/dev/rakudo/test/C.pm (C)' (from
'/home/froggs/dev/rakudo/test/B.pm (B)')

@p6rt
Copy link
Author

p6rt commented Jun 7, 2016

From @LLFourn

This is covered in​:

https://rt.perl.org/Public/Bug/Display.html?id=128156

On Tue, Jun 7, 2016 at 6​:24 AM Tobias Leich <perl6-bugs-followup@​perl.org>
wrote​:

# New Ticket Created by Tobias Leich
# Please include the string​: [perl #​128332]
# in the subject line of all future correspondence about this issue.
# <URL​: https://rt-archive.perl.org/perl6/Ticket/Display.html?id=128332 >

I've got the following files​:

~/dev/rakudo/test$ tree .
.
├── A.pm
├── B.pm
└── C.pm

$ cat A.pm
use C :foo;
use B;

$ cat B.pm
use C :foo;

$ cat C.pm
use v6;
use nqp;
use NativeCall;
my class CStruct is repr('CStruct') is export(​:foo) { }
# my class CStruct is repr('CStruct') { }

When I run this code once​:
$ perl6 -I. -MA -e ''

All is fine... But when I run it again after adding an empty line to
file C.pm I get​:
perl6 -I. -MA -e ''
===SORRY!===
Missing or wrong version of dependency
'/home/froggs/dev/rakudo/test/C.pm (C)' (from
'/home/froggs/dev/rakudo/test/B.pm (B)')

@p6rt
Copy link
Author

p6rt commented Jun 7, 2016

The RT System itself - Status changed from 'new' to 'open'

@p6rt
Copy link
Author

p6rt commented Jun 7, 2016

From @LLFourn

I've tested both issues with the branch​:

issue #​1 (changing dependency contents leads to goofed re-precomp) -- Still
broken
issue #​2 (collisions on timestamps leads to goofed re-precomp) -- Fixed!

for reference another example of issue #​1 was reported recently​:

https://rt.perl.org/Public/Bug/Display.html?id=128332

On Tue, May 24, 2016 at 8​:17 PM mt1957 <mt1957@​gmail.com> wrote​:

Hi,

Another symptom from the same bug. After modifying a sub module I get an
error like the following;

Type 'Semi-xml​::Sxml' is not declared
at /home/marcel/Languages/Perl6/Projects/Semi-xml/t/108-xml.t​:43
------> my Semi-xml​::Sxml⏏ $x .= new;
Malformed my
at /home/marcel/Languages/Perl6/Projects/Semi-xml/t/108-xml.t​:43
------> my Semi-xml​::⏏Sxml $x .= new;

Then, after modifying the top module (just a simple say), everything
runs ok again.

This is Rakudo version 2016.04-209-gdb10a01 built on MoarVM version
2016.04-134-g9879233

Greetings,
Marcel

@p6rt
Copy link
Author

p6rt commented Jun 10, 2016

From @LLFourn

another one​:

https://rt.perl.org/Public/Bug/Display.html?id=128156

On Tue, Jun 7, 2016 at 9​:29 PM Lloyd Fournier <lloyd.fourn@​gmail.com> wrote​:

I've tested both issues with the branch​:

issue #​1 (changing dependency contents leads to goofed re-precomp) --
Still broken
issue #​2 (collisions on timestamps leads to goofed re-precomp) -- Fixed!

for reference another example of issue #​1 was reported recently​:

https://rt.perl.org/Public/Bug/Display.html?id=128332

On Tue, May 24, 2016 at 8​:17 PM mt1957 <mt1957@​gmail.com> wrote​:

Hi,

Another symptom from the same bug. After modifying a sub module I get an
error like the following;

Type 'Semi-xml​::Sxml' is not declared
at /home/marcel/Languages/Perl6/Projects/Semi-xml/t/108-xml.t​:43
------> my Semi-xml​::Sxml⏏ $x .= new;
Malformed my
at /home/marcel/Languages/Perl6/Projects/Semi-xml/t/108-xml.t​:43
------> my Semi-xml​::⏏Sxml $x .= new;

Then, after modifying the top module (just a simple say), everything
runs ok again.

This is Rakudo version 2016.04-209-gdb10a01 built on MoarVM version
2016.04-134-g9879233

Greetings,
Marcel

@p6rt
Copy link
Author

p6rt commented Jun 10, 2016

From @LLFourn

https://rt.perl.org/Public/Bug/Display.html?id=128371 rather

On Sat, Jun 11, 2016 at 8​:38 AM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

another one​:

https://rt.perl.org/Public/Bug/Display.html?id=128156

On Tue, Jun 7, 2016 at 9​:29 PM Lloyd Fournier <lloyd.fourn@​gmail.com>
wrote​:

I've tested both issues with the branch​:

issue #​1 (changing dependency contents leads to goofed re-precomp) --
Still broken
issue #​2 (collisions on timestamps leads to goofed re-precomp) -- Fixed!

for reference another example of issue #​1 was reported recently​:

https://rt.perl.org/Public/Bug/Display.html?id=128332

On Tue, May 24, 2016 at 8​:17 PM mt1957 <mt1957@​gmail.com> wrote​:

Hi,

Another symptom from the same bug. After modifying a sub module I get an
error like the following;

Type 'Semi-xml​::Sxml' is not declared
at /home/marcel/Languages/Perl6/Projects/Semi-xml/t/108-xml.t​:43
------> my Semi-xml​::Sxml⏏ $x .= new;
Malformed my
at /home/marcel/Languages/Perl6/Projects/Semi-xml/t/108-xml.t​:43
------> my Semi-xml​::⏏Sxml $x .= new;

Then, after modifying the top module (just a simple say), everything
runs ok again.

This is Rakudo version 2016.04-209-gdb10a01 built on MoarVM version
2016.04-134-g9879233

Greetings,
Marcel

@p6rt
Copy link
Author

p6rt commented Jun 11, 2016

From @niner

I've golfed the test case for changing a dependency​:

cat test.sh
echo 'use C; use B;' > A.pm
echo 'use C;' > B.pm
echo 'use D;' > C.pm
touch D.pm
rm -Rf .precomp/ && perl6 -I. -MA -e '' && echo >> C.pm && perl6 -I. -MA -e ''

@p6rt
Copy link
Author

p6rt commented Jun 12, 2016

From @FROGGS

These patches fix and test the remaining issue​:
  rakudo/rakudo@d0a00164e9
  Raku/roast@716b94f7ff

1 similar comment
@p6rt
Copy link
Author

p6rt commented Jun 12, 2016

From @FROGGS

These patches fix and test the remaining issue​:
  rakudo/rakudo@d0a00164e9
  Raku/roast@716b94f7ff

@p6rt p6rt closed this as completed Jun 12, 2016
@p6rt
Copy link
Author

p6rt commented Jun 12, 2016

@FROGGS - Status changed from 'open' to 'resolved'

@p6rt
Copy link
Author

p6rt commented Jun 15, 2016

From @LLFourn

@​FROGGS for me that test passes with
rakudo/rakudo@491af6034d
<rakudo/rakudo@491af6034d4bc328a9a266528e5ec19f132b2611>
(without
your fix).
Your commit did fix the issue of the file's content changing (and therefore
the SHA) causing re-precompilation to be broken​:

Tests for that​:
Raku/roast@cef40ed

\o/ precomp works again. Thanks all!

On Mon, Jun 13, 2016 at 3​:22 AM Tobias Leich via RT <
bugs-comment@​bugs6.perl.org> wrote​:

These patches fix and test the remaining issue​:
rakudo/rakudo@d0a00164e9
Raku/roast@716b94f7ff

@p6rt p6rt added the testneeded label Jan 5, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant