Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 21 20:01
    ddfreyne commented #1483
  • Jan 21 19:59
    da2x commented #1483
  • Jan 21 19:57
    da2x opened #1483
  • Jan 21 17:24
    joerggollnick commented #1479
  • Jan 21 15:27
    da2x opened #1482
  • Jan 21 07:59
    ddfreyne opened #1481
  • Jan 21 07:58

    ddfreyne on rm-safe-level

    Remove safe_level support in ER… (compare)

  • Jan 21 07:57

    ddfreyne on rm-safe-level

    Remove safe_level support in ERB (compare)

  • Jan 21 00:09
    bobbens edited #1475
  • Jan 20 22:29
    da2x commented #1475
  • Jan 10 07:21
    cdlm commented #1480
  • Jan 10 01:53
    cup edited #1480
  • Jan 10 01:52
    cup opened #1480
  • Jan 08 17:06
    blinry commented #1471
  • Jan 08 11:14
    iay opened #1479
  • Dec 26 2019 10:14
    ddfreyne labeled #1477
  • Dec 26 2019 10:14
    ddfreyne labeled #1478
  • Dec 26 2019 10:14
    ddfreyne closed #1478
  • Dec 26 2019 10:14
    ddfreyne commented #1478
  • Dec 26 2019 03:41
    cup opened #1478
Denis Defreyne
@ddfreyne
@thosgood Hmmm… what if you configure nanoc.yaml and set remote to be origin (rather than git@github…)
git push will use the default remote (which would be origin) so maybe there’s some remote funkiness
Tim Hosgood
@thosgood
yes!
that fixed it perfectly, thank you!
must have been some mistake in how I set the remote
Andri Möll
@moll
I've noticed nanoc live --live-reload often ends in a situation where an edit causes a refresh, but the refresh happens before the new version of a page has been written to disk. Live-reload therefore gets shown a 404 page. Apart from just playing with the LiveReload delay option, do we have any smarter solutions?
Andri Möll
@moll
I'm using guard-nanoc for the live command.
Seems https://github.com/ddfreyne/adsf/blob/master/adsf-live/lib/adsf/live/watcher.rb is responsible for setting up LiveReload, and it hard-codes the delay to 0, along with hard-coding the port.
I think the port should be randomized to permit multiple instances of LiveReload to be running on the same computer.
Denis Defreyne
@ddfreyne
Hmmmmm… but livereload is supposed to reload when it detects changes to the compiled site only, so maybe something else is going wrong here
Andri Möll
@moll
How does Nanoc write an output file? Straight over the existing file or does it use a temporary file with renaming?
Denis Defreyne
@ddfreyne
@moll It uses a temporary file
Andri Möll
@moll
That would explain why it's triggering LiveReload before it's finished.
If that temporary file is placed in the same output directory.
Nanoc could place it in tmp and rename from there. That'd fix early reloads. Better though probably to hook into LiveReload and trigger it once compilation is finished, but I don't know whether the LiveReload library permits that.
Denis Defreyne
@ddfreyne
Hmm no, it’s written into /tmp/* and then moved into place
Andri Möll
@moll

Mkay, I'll run inotifywait next time I'm working on my site and confirm what Nanoc's doing. Perhaps a directory gets touched (by the kernel) first, triggers inotify and LiveReload catches that.

You're claiming Nanoc's renaming files over existing files, right? It's not a delete-then-move action?

Presumably at some point the output file has to be missing entirely, otherwise the web server wouldn't report a 404.
Denis Defreyne
@ddfreyne
Hmmm, it should be in place!
This is pretty weird.
Andri Möll
@moll

Curiously, I'm seeing a delete happen ...

19:51:36 DELETE public/articles/foo/index.html
19:51:36 CREATE public/articles/foo/index.html
19:51:36 MODIFY public/articles/foo/index.html
19:51:36 CLOSE_WRITE,CLOSE public/articles/foo/index.html

This with:

inotifywait --recursive --monitor public --format "%T %e %w%f" --timefmt "%T" -e modify -e attrib -e close_write -e moved_to -e moved_from -e move -e move_self -e create -e delete -e delete_self
And that triggered a race which LiveReload won, reloading before the create and coming up with "File not found".
Andri Möll
@moll
I think I know why this is happening. Nanoc's not using renaming, it's using hard linking.
https://github.com/nanoc/nanoc/blob/4.10.1/nanoc/lib/nanoc/base/services/item_rep_writer.rb#L59 calls FileUtils.ln with force: true. Force, as per Ruby's implementation, just unlinks it before creating a new link:
https://ruby-doc.org/stdlib-2.4.1/libdoc/fileutils/rdoc/FileUtils.html#method-c-ln
That explains the delete inotify event before create and explains the race condition.
Andri Möll
@moll
Why's Nanoc using linking or copying there in the first place? Wouldn't it be fine to move the temporary file? I suppose it's because you were trying to optimize by hard linking binary files.
..given that in one branch of the code temp_path is set to not a true temporary path, but the actual filename from the content directory.
Denis Defreyne
@ddfreyne
@moll The hardlinking’s purpose is to improve disk usage
Moving stuff in place can work under certain conditions, but not all
(e.g. the input file in content/ might be hardlinked to what is in output/!)
Andri Möll
@moll
Are you saying that non-binary items are hardlinked from content? From my brief reading I got the impression compiled items are just written to tmp, then hardlinked, then removed from tmp.
Denis Defreyne
@ddfreyne
Non-binary items are not hardlinked from content.
Andri Möll
@moll
Are compiled items stored somewhere aside from the output directory? I didn't notice any in my tmp...
Denis Defreyne
@ddfreyne
The intermediate compiled files are stored in the OS’ temporary directory (/tmp somewhere)
Andri Möll
@moll
For a long time or just during a single compile?
Denis Defreyne
@ddfreyne
During a single compilation
Andri Möll
@moll

Got it. Then it seems I guessed correctly before:

Wouldn't it be fine to move the temporary file? I suppose it's because you were trying to optimize by hard linking binary files.

I'm not sure how hard linking helps for compilable items then.
Denis Defreyne
@ddfreyne
For compileable items?
Do you mean textual, or binary?
Andri Möll
@moll
Presumably anything that's not identical to content, but particularly text files.
Denis Defreyne
@ddfreyne
I think what surprises me is that the DELETE + CREATE + MODIFY happens slow enough that the browser actually sees the file as missing
Andri Möll
@moll
It seems there's a pretty good opportunity for optimizing "compilation" of passed through binary items, too — not to compare them byte by byte with FileUtils.identical, but check if they're hard links. If I change the hard linked file in content, it's guaranteed to already match in output.
Yeah, I'm seeing it approximately every 10 updates. Fast computer, what can I say. :P
Denis Defreyne
@ddfreyne
Also makes me wonder whether Nanoc could create a hard link temporarily in ./tmp/nanoc (inside the site dir) and then move it in place (which would be atomic)…
Andri Möll
@moll
Still, how does a hard link help there?
There's one index.html compiled to /tmp. That's not around after the compile ends. Why not move it immediately?
Denis Defreyne
@ddfreyne
nanoc/nanoc#1321 Here’s the MR that introduced hard links
Andri Möll
@moll
I find his hard linking proposal to be sensible for files that are identical. For the changed items, I'm still puzzled.
Identical between content and output, that is.
Even the copy fallback seems wasteful if I understand the flow correctly. Given a compiled file (like Markdown -> HTML) temp_path referes to a file that's deleted after the method finishes. That file shouldn't be copied and then deleted, it should only be moved.