Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 06:26
    ariesclark commented #3114
  • 00:43
    JounQin opened #4990
  • 00:43
    JounQin labeled #4990
  • 00:31
    zkochan commented #4868
  • 00:18
    zkochan commented #4868
  • 00:11
    JounQin closed #4983
  • 00:11
    JounQin commented #4983
  • 00:05
    zkochan commented #4983
  • 00:04
    zkochan commented #4983
  • 00:00
    JounQin commented #4983
  • Jul 06 23:51
    zkochan commented #4983
  • Jul 06 23:50
    JounQin commented #4983
  • Jul 06 23:41
    zkochan commented #4980
  • Jul 06 23:41
    zkochan commented #4983
  • Jul 06 22:45
    JounQin commented #4980
  • Jul 06 22:40
    JounQin commented #4983
  • Jul 06 22:40
    JounQin commented #4983
  • Jul 06 22:39
    JounQin commented #4983
  • Jul 06 22:39
    JounQin commented #4983
  • Jul 06 22:22
    jondlm commented #4868
Zoltan Kochan
@zkochan
if that would help, we can add an option to use symlinks on Windows.
they are not used by default because symlinks on Windows require admin permissions
but if your CI is a Windows, you can probably configure it to work fine with symlinks
Zoltan Kochan
@zkochan

@/all pnpm version 1.3 is out

It is 5% faster than 1.2 and has a nice new summary of added/deleted root dependencies:

Imgur

all the changes here

Fox George Penguin
@olstenlarck
@zkochan cool, that summary seems great! :)
Zoltan Kochan
@zkochan
:tada:
Zoltan Kochan
@zkochan

pnpm list implemented in v1.5.

The new command was implemented in pnpm-list and required in the main package

Andrei Neculau
@andreineculau

[WARN] long wall of text coming up

I didn't know about pnpm until today actually, but it sure sounds great :clap: .
Before I give it a shot in my team, I wonder if you have already solved the following problem: creating a deployable bundle that is space-optimized when unpacked (the archive compressor would take care of most of the duplication, but we have limited disk space when unpacking the bundle).

A bit of context is that today we do something similar to pnpm (i.e. we don't use npm install the regular way, and use symlinks to gain speed) and at build-artifact step, we actually do a full copy and call npm dedupe <rant>npm@3 actually, since that's the only one that still works properly...</rant>
AFAICS using pnpm, instead of our install setup, should work just as well, and then we could still do a full copy (i.e. no hardlinks) and call npm dedupe.
But do you (pnpm users or pnpm itself) have anything to aid the need for a deduped node_modules ? Thanks!

Zoltan Kochan
@zkochan

Hi! We're happy you have discovered pnpm!

pnpm would be a perfect choice for your team if you are currently hacking npm with symlinks. pnpm uses a symlinked node_modules structure and it hard links a package/version only once to a project, then it
creates symlinks to create the dependency tree. Read more about the symlinked node_modules here: https://github.com/pnpm/pnpm/blob/master/docs/symlinked-node-modules-structure.md

pnpm doesn't need dedupe because the structure it creates is always the same. It is very deterministic. Nothing that is not needed is added or is left in node_modules.

Nik
@nikdojo
Hi guys! I'm testing pnpm for Ember CLI app, the speed is fantastic. But there are some issues with dependencies that are not in "dependencies" section of the app. For example, Cannot find module 'route-recognizer' from '/my-app/node_modules/.registry.npmjs.org/ember-cli-pretender/1.0.1/node_modules/pretender', so it can't find deps in a flat structure of modules. I read all the docs I could find and issue tickets for pnpm/ember-cli, but seems nothing points on this. Does anybody have experience using pnpm with Ember, or have any idea where the problem is? Appreciate any help, thanks.
Zoltan Kochan
@zkochan

Hi, this is just a symlink, dependencies of pretender won't be resolved from this path /my-app/node_modules/.registry.npmjs.org/ember-cli-pretender/1.0.1/node_modules/pretender. The real location of pretender is used to resolve route-recognizer, which is probably /my-app/node_modules/.registry.npmjs.org/pretender/1.5.1/node_modules/pretender

and there should be a symlink to route-recognizer in /my-app/node_modules/.registry.npmjs.org/pretender/1.5.1/node_modules/route-recognizer

If you can file an issue with steps to repro, we'll be able to help. Ideally with an example repo or a package.json

Nik
@nikdojo
Dependencies: https://pastebin.com/NMQmsKEF
All symlinks within modules structure seem to be fine.
Zoltan Kochan
@zkochan
when does the error happen?
when running some command?
Nik
@nikdojo
Just run the app with ember serve for example
Zoltan Kochan
@zkochan
with the globally installed ember?
Nik
@nikdojo
not necessary, u can install locally and then run with ./node_modules/.bin/ember serve
Zoltan Kochan
@zkochan
ok, I'll try later
Nik
@nikdojo
thx a lot!
Zoltan Kochan
@zkochan
np
Nik
@nikdojo

@zkochan please download the basic Ember app here: https://drive.google.com/file/d/0B6psKNv4yBIcU3VXTWpld2dqQmM/view?usp=sharing

Then:
pnpm i && bower i && ./node_modules/.bin/ember serve

Should see the error.

Zoltan Kochan
@zkochan
ok, cool.
Zoltan Kochan
@zkochan
ok, I've seen this error before in a few projects. This is because the resolve package does not resolve dependencies the way Node does... We'll have to create an issue there. There is actually an alternative to resolve that works correctly - resolve-from by Sindre Sorhus
Andrei Neculau
@andreineculau
@zkochan thanks for the explanation/link! Nothing left but to try it out then!
Nik
@nikdojo
@zkochan I took a look at resolve-from. It seems it doesn't make any difference in terms of resolving pnpm-structure. You see, the thing is both resolve & resolve-from use node-module-path method to get a list of possible directories for a module. But none of them provide a possibility to get a proper dir name for node_modules/.registry.npmjs.org/MODULE_NAME/VERSION/... 'coz no module version is available at this time. And there is no symlink to it in the current folder of ember-cli-pretender.
Nik
@nikdojo
@zkochan I managed to change resolve to search for deps in flat structure, but there is another issue appear. For example, broccoli-persistent-filter has dep "async-disk-cache": "^1.2.1", but there is only async-disk-cache/1.3.2/ exists (and no symlink for 1.2.1).
Zoltan Kochan
@zkochan
@vjpr maybe you'll have some ideas about these issues? I can look into it later today
Nik
@nikdojo
@here hey guys, who interested in conversation abt Ember apps & pnpm — please follow here: pnpm/pnpm#857
Zoltan Kochan
@zkochan
created an issue at resolve substack/node-resolve#130
and at resolve-from sindresorhus/resolve-from#6
Andrei Neculau
@andreineculau
question: if one does not want shrinkwrap.yaml created, is there a flag to pass on to pnpm?
Zoltan Kochan
@zkochan
why would someone not want it? All node package managers have lockfiles now because they are awesome!
but there is no such flag, if you really don't want it, you have to put it to gitignore
I don't know if you checked it, but shrinkwrap.yaml looks really nicely on git diff
it precisely shows what deps were changed where
IMHO, the most readable lockfile diffs and nicer for CR than yarn.lock and package-lock.json
but again, there is no way to disable it because pnpm's algorithm relies heavily on the shrinkwrap.yaml file. So if you don't want it in your repo, just ignore it
Andrei Neculau
@andreineculau
:) maybe they are awesome, not contesting that, but maybe some people don't want to be awesome :p ---
  1. generally speaking, committing generated content on a routine basis like adding/updating/removing deps is plain wrong (generating some images for instance, committing them, and maybe regenerating them one year later is a diff thing)
  2. whoever is working towards a clean, atomic commit history (i.e. rewrites history in the local/topic branch) increased their chances to get merge conflicts, which they didn't create in the first place
  3. most of the times when no.2 happens, the feeling of "whatever" takes over so the shrinkwrap.yaml becomes out of sync with its source (package.json) leading to other errors or making it impossible to git checkout a commit and run with it
  4. it makes total sense to have the shrinkwrap part of the distribution package yes i.e. have a stable deployment, both when dependencies are bundled in the distro, and when they are not
that said, i noticed the love and care that was given in the design of the file :clap: and i guessed the diff was a factor
.gitignore-ing the file is not enough i think e.g. what happens if someone updates the package.json manually, upgrades a dep, and then runs pnpm install - the shrinkwrap.yaml will prevail, right? or does pnpm check modification time :clap: ?
Zoltan Kochan
@zkochan
it will work. there is a specifiers field in shrinkwrap.yaml and it is matched with the ones in package.json
package.json is the boss
Andrei Neculau
@andreineculau
:clap:
Zoltan Kochan
@zkochan
have you read this article by James Kyle? https://yarnpkg.com/blog/2016/11/24/lockfiles-for-all/
Andrei Neculau
@andreineculau
i will give it a try (generate and commit), i come back with feedback. can very well be more claps. fwiw thanks to everyone that started and contributed to pnpm. it's a breath of fresh air to see simplicity in the nodejs world :smile:
Zoltan Kochan
@zkochan
:tada:
Andrei Neculau
@andreineculau

@zkochan missed your link. yes, i read it, pretty much when it was published. sorry if it doesn't change my stand.

i hope we can at least agree that issues like these are not just smoke - yarnpkg/yarn#2155 yarnpkg/yarn#1776 and similar thread on all package managers that have a lockfile. And most of them end up saying "yes, commit it, because it's going to make your life easier, but when you get merge conflicts, delete it and regenerate it" --- and on each one there's a person that gets the lightbulb "hold on.. if I have version A from 1995, version B from 2000, I want to merge them them, get a conflict, I delete and regenerate in 2005, doesn't it mean I will basically upgrade all sorts of sub-dependencies and loose the lock on them?"

Zoltan Kochan
@zkochan
I can't tell how the shrinkwrap.yaml currently behaves when merged because I don't use it currently in a team. The company I am working at uses npm4.
I think however it should be pretty good at merges because it doesn't have commas like npm's json
and it doesn't have specs in one line like yarn
yarn has something like foo@^1.0.0, foo@^1.1.0: in a line. Which can differ on merge, in shrinkwrap.yaml always blocks differ, not lines
Zoltan Kochan
@zkochan
We have an interesting discussion about the possibility to override subdependencies: pnpm/pnpm#651
Andrei Neculau
@andreineculau
is pnpm install X supposed to be different than pnpm add X ? (install doesn't do uninstall and install, add does)