The personal, minimalist, super-fast, database free, bookmarking service - community repo
It seems I spoke too soon. shaarli is working OK, all I see in the server log is warnings that shorturls or thumbnails are undefined. When I try to export the 27k+ bookmarks in json format using this trick: /usr/bin/php -r 'print(json_encode(unserialize(gzinflate(base64_decode(preg_replace("!./\ (.+) */.*!", "$1", file_get_contents("20180902.shaarlidatastore.php"))), 67108864))));'
I now always get this error:
PHP Warning: gzinflate(): data error in Command line code on line 1
no matter how I set the optional length parameter of gzinflate . The datastore.php file is 7287011 bytes
What can it be? May the reason be that I imported bookmarks without shorturl or thumbnail, or that I had started synchronizing them but then stopped? Is there any way to get around this problem
update/rewording of the problem I am having:
shaarli 0.10.2 on php7/ubuntu 16.04 x86_64seems to work perfectly with 27k+ bookmarks... while using/generating a CORRUPTED datastore.php
Details: the software works. I can display bookmarks, filter them, add them with the bookmarklet, edit, see the picwall... everything. And the only thing I see in the apache error log file is the occasional PHP Notice: "Undefined index: shorturl in /var/www/html/shaarli/tmp/linklist.b91ef64efc3688266305ea9b42e5017e.rtpl.php on line 282,
BUT: this hack to export the shaarli bookmarks (which is essential for me)
php -r 'print(json_encode(unserialize(gzinflate(base64_decode(preg_replace("!./\ (.+) */.*!", "$1", file_get_contents("/var/www/html/shaarli/data/datastore.php")))))));'
constantly fails saying "PHP Warning: gzinflate(): data error in Command line code on line 1", no matter what I set the length parameter of gzinflate (or gzdecode) to. Even if that is the same gzinflate function that is working perfectly when called from the browser/apache. And if I copy datastore.php to another file, remove the php tags, convert it to a test.gz file with the base64 tool, indeed gunzip says that that file is NOT a gzipped file. Ditto for unzip.
Any clue to what is happening? Thanks
php -r 'print(json_encode(unserialize(gzinflate(base64_decode(preg_replace("!./\ (.+) */.*!", "$1", file_get_contents("/var/www/links/data/datasphp")))))));' on my install and it fails as well. I would double check that the preg_replace doesn't mangle actual parts of the file (what re you trying to replace with it?)
A user of Shaarli Material theme has a strange issue which seems related to Shaarli itself, even though it apparently doesn't happen with the default theme.
https://github.com/kalvn/Shaarli-Material/issues/90#issuecomment-453805952
Did you already have such error before? Any idea where it could come from? I wasn't able to reproduce so far.
Sadly there's no such guide, and the templating engine used by Shaarli (RainTPL3) is both unmaintained and quite poorly documented...
A switch to Twig was mentioned a while back, and was put aside as it would imply 1/ refactoring the rendering engine and 2/ supporting multiple templating backends not to break existing themes
most refactoring efforts are tracked on shaarli/Shaarli#324 and this project board; there have been great improvements made since Shaarli was forked a while back (e.g. unitary testing, namespacing, documentation, separation of concerns, etc.) but development is still somewhat hindered due to (IMHO):
- using a serialized array as a database: this leads to quite convoluted data access/update code, not to mention separation of concerns and database migrations
- the lack of a proper data model also prevents from having multi-user instances
- NIH: homemade authentication and session management, homemade syndication, etc. we're slowly getting to the point where we can isolate such components and maybe start replacing them in favor of standard/community-supported components
on top of this, we're careful not to break existing installations during upgrades, and keep backwards compatibility as much as possible, though this is not always feasible (e.g. with 3rd-party plugins and themes)
ideally it'd be really nice to be able to rely on actively maintained 3rd-party libraries for all generic stuff (authentication, templating, routing, etc.) and build upon them to provide features dedicated to managing a collection of bookmarks/notes
The first point, I think, is why at least a couple of people have advocated for using a SQL database (like SQLite) instead of a flat file. Same with the ad-hoc data model; that would be a good idea even for Shaarli's single user architectre.
I am one of them. +100 to that proposal. I've been using shaarli for ~7/8 months now, to handle +15k bookmarks, increasing every week, and can now confirm that a) shaarli remains the ONLY usable alternative even for power users, but b) its "flat file" architecture greatly limits both performances and flexibility. If one could query/update the bookmarks database with standard and fast SQLite queries, the usefulness of shaarli would increase tenfold, really.
Now, I do know that switching to SQLite would be a BIG task, and I have not enough php/SQLite skills to actually hack/rewrite the source code myself. BUT if someone else can do that, I believe I could do a pretty good job as tester. Besides having previous experience on that kind of work, as I said I have a pretty big, real-world shaarli database already, and making a separate install of shaarli just to make automatic tests on that would be no problem. So count me in for that.
tag improvements: https://github.com/shaarli/Shaarli/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+label%3AtagsSpaces in tags (or the ability to set a delimiter) is a recurring request theme :)
The SQL aspect has been bugging me for a while, this is something I'd very much like to see happening at some point! I'll try to start experimenting with Doctrine when I get enough spare time.
Regarding testing, I think something that would be greatly beneficial for developers and community alike would be to add functional tests with browser automation, e.g. with Selenium and/or RobotFramework. We're using both at work and they nicely complement unitary testing ;-)
I spent a lot of last night playing around with editing the default theme for Shaarli. What about an HTML5 boilerplate theme (https://html5boilerplate.com/) for Shaarli, to make it easier to write new themes? Pretty much all of the ones out there don't work anymore due to the namespacing changes, and such a thing would make it much easier to rebuild the Shaarli ecosystem.
/one from your configuration file? url = must be the base URL of your shaarli instance. See https://python-shaarli-client.readthedocs.io/en/latest/user/configuration.html#example.
Enable REST API is checked in Shaarli settings, also make sure your server is properly configured (including URL rewriting support)
@virtadpt about "interfacing shaarli with ArchiveBox", which I only saw today: I believe it could work this way:
1) external script/cronjob a) gets with shaarli_client the list of bookmarks to make copies of, b) passes that list to archivebox and c) saves each file in a folder called $BOOMARKS_HOME/$ID/index.html where $ID is the identifier of that bookmark inside shaarli.
2) then, all shaarli needs to do is to add, ONLY when user is logged in, a link/button to $BOOMARKS_HOME/$ID/index.html, close to where the "edit" icon of the same bookmark is. Theoretically, this should be very, very little code, no?
Is this correct? If it is... I know how to do 1) myself, hope to actually do it in the next weeks. But I honestly have no clue on how to do 2). I don't care if there is no gui interface, i.e. I don't mind patching the shaarli source code to add the php commands that display that extra link to logged in users... but I would really need someone to write down that code for me, and tell me where to put it. Once I have that info, I can go and test myself, and report, so someone else can make a usable plugin.
@virtadpt the problem with that approach is that the RSS feed would not include private bookmarks (incidentally, this is one more case/reason why having the bookmarks as one SQLite db would be waaaay better, but that is another issue, of course). And in any case, the real problem I have and need help with is part 2 of my earlier comment: how do I hack shaarli's source code to make it display,ONLY to logged-in user, one extra link with the same bookmark ID?
Concrete example: if I look right now at my own shaarli installation, the latest (private!) bookmark includes an icon that is a link to https://example.com/?edit_link=289. Well, all I need to know is how to change the php code so that, right after that icon/link, it displays another one pointing to https://example.com/WEBARCHIVE/289/index.html, which would be the copy of the same page created by archivebox . For the record, I ALREADY tried to do this, but after a couple of hours or so of looking at the code I had to stop. My limit, of course. That is why I ask for help