Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible precomp issue: crash with is a directory, cannot do '.open' on a directory pointing to a sha1 hash of a dir #6422

Closed
p6rt opened this issue Aug 2, 2017 · 7 comments

Comments

@p6rt
Copy link

p6rt commented Aug 2, 2017

Migrated from rt.perl.org#131831 (status was 'rejected')

Searchable as RT131831$

@p6rt
Copy link
Author

p6rt commented Aug 2, 2017

From @zoffixznet

The full issue and some discussion can be seen on zef's Issue ugexe/zef#190

After working around --ll-exception, we managed to dump full stack trace.

It points to this line in CURI https://github.com/rakudo/rakudo/blob/7599e0c3ffb2c7ef29c2994c7d212360459f14fe/src/core/CompUnit/Repository/Installation.pm#L424

It's checking for `.e` but probably should be checking for `.f`. I'm not sure if that's all to the issue (probably not) and
perhaps someone more familiar with the codebase would know WHY the $lookup ends up being a directory instead of the
expected file​:

$ cd $(mktemp -d); mkdir Exceptions; echo 'class Exceptions​::Foo { method process ($e) { say "$e.message() $e.backtrace.full()"; False } }' > Exceptions/Foo.pm6; RAKUDO_EXCEPTIONS_HANDLER=Foo perl6 -I. -MExceptions​::Foo /usr/share/perl6/vendor/bin/zef
'/usr/share/perl6/vendor/short/34674DE0B49051C42E89F10D51FE788C3C3A3816' is a directory, cannot do '.open' on a directory in method throw at SETTING​::src/core/Exception.pm line 57
  in block at SETTING​::src/core/CompUnit/Loader.pm line 25
  in block at SETTING​::src/core/CompUnit/Loader.pm line 23
  in any at /usr/share/perl6/runtime/CORE.setting.moarvm line 1
  in method throw at SETTING​::src/core/Exception.pm line 54
  in method throw at SETTING​::src/core/Failure.pm line 44
  in method Str at SETTING​::src/core/Failure.pm line 85
  in method lines at SETTING​::src/core/Cool.pm line 169
  in method lines at SETTING​::src/core/IO/Path.pm line 629
  in block at SETTING​::src/core/CompUnit/Repository/Installation.pm line 425
  in block at SETTING​::src/core/CompUnit/Repository/Installation.pm line 424
  in method matching-dist at SETTING​::src/core/CompUnit/Repository/Installation.pm line 421
  in method need at SETTING​::src/core/CompUnit/Repository/Installation.pm line 503
  in method need at SETTING​::src/core/CompUnit/Repository/Installation.pm line 543
  in method need at SETTING​::src/core/CompUnit/Repository/Installation.pm line 543
  in method need at SETTING​::src/core/CompUnit/Repository/FileSystem.pm line 148
  in any load_module at src/Perl6/World.nqp line 1218
  in any do_pragma_or_load_module at src/Perl6/World.nqp line 1148
  in any statement_control​:sym<use> at src/Perl6/Grammar.nqp line 1640
  in any statement_control at /usr/share/nqp/lib/Perl6/Grammar.moarvm line 1
  in any statement at src/Perl6/Grammar.nqp line 1292

@p6rt
Copy link
Author

p6rt commented Aug 2, 2017

From @zoffixznet

Also some discussion on IRC​: https://irclog.perlgeek.de/perl6/2017-08-02#i_14957916

@p6rt
Copy link
Author

p6rt commented Aug 2, 2017

From jonathon@manjaro.org

Just as a brief comment, a switch from `.e` to `.f` in only that file, e.g.

sed -i '424s|\.e|\.f|' src/core/CompUnit/Repository/Installation.pm

leads to an error during `make install`​:

```
mkdir -p -- /dev/shm/rakudo/pkg/rakudo/usr/bin
mkdir -p -- /dev/shm/rakudo/pkg/rakudo/usr/share/nqp/lib/Perl6
cp -- blib/Perl6/ModuleLoader.moarvm blib/Perl6/World.moarvm blib/Perl6/Grammar.moarvm blib/Perl6/Ops.moarvm blib/Perl6/Actions.moarvm blib/Perl6/Optimizer.moarvm blib/Perl6/Pod.moarvm blib/Perl6/Compiler.moarvm blib/Perl6/Metamodel.moarvm blib/Perl6/BOOTSTRAP.moarvm blib/Perl6/DebugPod.moarvm /dev/shm/rakudo/pkg/rakudo/usr/share/nqp/lib/Perl6
mkdir -p -- /dev/shm/rakudo/pkg/rakudo/usr/share/perl6/lib
mkdir -p -- /dev/shm/rakudo/pkg/rakudo/usr/share/perl6/runtime
cp -- CORE.setting.moarvm CORE.d.setting.moarvm RESTRICTED.setting.moarvm /dev/shm/rakudo/pkg/rakudo/usr/share/perl6/runtime
cp -- perl6.moarvm perl6-debug.moarvm /dev/shm/rakudo/pkg/rakudo/usr/share/perl6/runtime
mkdir -p -- /dev/shm/rakudo/pkg/rakudo/usr/share/perl6/runtime/dynext
cp -- dynext/libperl6_ops_moar.so /dev/shm/rakudo/pkg/rakudo/usr/share/perl6/runtime/dynext
./perl6-m tools/build/upgrade-repository.pl /dev/shm/rakudo/pkg/rakudo/usr/share/perl6
./perl6-m tools/build/upgrade-repository.pl /dev/shm/rakudo/pkg/rakudo/usr/share/perl6/vendor
./perl6-m tools/build/upgrade-repository.pl /dev/shm/rakudo/pkg/rakudo/usr/share/perl6/site
./perl6-m tools/build/install-core-dist.pl /dev/shm/rakudo/pkg/rakudo/usr/share/perl6
===SORRY!===
Could not find CompUnit​::Repository​::Staging at line 0 in​:
  /dev/shm/rakudo/pkg/rakudo/usr/share/perl6
  /usr/share/perl6
  CompUnit​::Repository​::AbsolutePath<830429318560>
  CompUnit​::Repository​::NQP<830450738400>
  CompUnit​::Repository​::Perl5<830450738440>

  in block <unit> at tools/build/install-core-dist.pl line 26

make​: *** [Makefile​:620​: m-install] Error 1
```

@p6rt
Copy link
Author

p6rt commented Aug 5, 2017

From @niner

The .e there is absolutely correct as depending on the repository format version $lookup may be a file or directory. We switched to a directory with version 1. So the .f will fail on a current repo version which explains the failure in install-core-dist.pl

The real question is not why $lookup ends up being a directory (that's the normal case) but why we don't detect that we are dealing with a version 2 repo.

The PKGBUILD script posted on ugexe/zef#190 gives the answer to that​:

  rm -f "$pkgdir/usr/share/perl6/vendor/version"

Of course, by deleting that file, you rob rakudo of a vital piece of information. Without that it runs on the wrong assumption that it's dealing with a version 0 repository and that $lookup must be a file.

There's really nothing we could or should do here. As much as I'd like us to throw a more useful error message here, that's simply not possible without slowing down all normal uses. And trying to make rakudo resilient against random files missing sounds like an uphill battle.

Please just remove the offending lines from the script. This, too looks highly suspicious​:
msg2 'Removing redundant precomp file dependencies...'
  _precomp=($(pacman -Qqg perl6 | grep -v zef | pacman -Qql - | grep -E 'dist|precomp' || true))
  for _pc in "${_precomp[@​]}"; do
  [[ -f "$pkgdir/$_pc" ]] && rm -f "$pkgdir/$_pc"
  done

Considering that the spec file for openSUSE does not delete anything, I suggest starting with an absolute minimum of a build script. The install-dist.pl call may be quite sufficient. openSUSE does have quite strict packaging guidelines, too.
https://build.opensuse.org/package/view_file/devel:languages:perl6/perl6-Inline-Perl5/perl6-Inline-Perl5.spec?expand=1

@p6rt
Copy link
Author

p6rt commented Aug 5, 2017

The RT System itself - Status changed from 'new' to 'open'

@p6rt
Copy link
Author

p6rt commented Aug 5, 2017

@niner - Status changed from 'open' to 'rejected'

@p6rt p6rt closed this as completed Aug 5, 2017
@p6rt
Copy link
Author

p6rt commented Aug 6, 2017

From jonathon@manjaro.org

The missing 'version' file looks to be the issue, and explains why zef worked when installing under under 'site' but not 'vendor'​: during my testing I think I managed to delete the version file from vendor...

Oops. That means I've created a load of noise (but hopefully someone might benefit from it down the line...).

This isn't related to the PKGBUILD file itself (where version was being deleted); that's a chroot/shadow dir so the removal would be to prevent a file conflict, not directly affect files on the filesystem. However, 'install-dist.pl' takes care of that so the section isn't needed anyway.

Thank you for looking into this. :)

On Sat, 05 Aug 2017 07​:12​:17 -0700, nine@​detonation.org wrote​:

The .e there is absolutely correct as depending on the repository
format version $lookup may be a file or directory. We switched to a
directory with version 1. So the .f will fail on a current repo
version which explains the failure in install-core-dist.pl

The real question is not why $lookup ends up being a directory (that's
the normal case) but why we don't detect that we are dealing with a
version 2 repo.

The PKGBUILD script posted on ugexe/zef#190
gives the answer to that​:

rm -f "$pkgdir/usr/share/perl6/vendor/version"

Of course, by deleting that file, you rob rakudo of a vital piece of
information. Without that it runs on the wrong assumption that it's
dealing with a version 0 repository and that $lookup must be a file.

There's really nothing we could or should do here. As much as I'd like
us to throw a more useful error message here, that's simply not
possible without slowing down all normal uses. And trying to make
rakudo resilient against random files missing sounds like an uphill
battle.

Please just remove the offending lines from the script. This, too
looks highly suspicious​:
msg2 'Removing redundant precomp file dependencies...'
_precomp=($(pacman -Qqg perl6 | grep -v zef | pacman -Qql - | grep
-E 'dist|precomp' || true))
for _pc in "${_precomp[@​]}"; do
[[ -f "$pkgdir/$_pc" ]] && rm -f "$pkgdir/$_pc"
done

Considering that the spec file for openSUSE does not delete anything,
I suggest starting with an absolute minimum of a build script. The
install-dist.pl call may be quite sufficient. openSUSE does have quite
strict packaging guidelines, too.
https://build.opensuse.org/package/view_file/devel:languages:perl6/perl6-
Inline-Perl5/perl6-Inline-Perl5.spec?expand=1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant