These are chat archives for UnderstandLingBV/Tuktu

22nd
Jul 2016
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:15
Ik ben bezig met het doortesten van de implementatie. Op dit moment zie ik echter geen data op 1 van de tuktu-servers... Is er een vertraging waaar ik rekening mee moet houden of is er iets niet goed?
blob
blob
Erik Tromp
@ErikTromp
Jul 22 2016 08:16
@ObjectivePartners English please
look in your Tuktu instance, the pixel flow has crashed
Illegal character in query at index 25: /Tuktu.gif/px?utm_source={{utm_source}}&utm_medium={{utm_medium}}&utm_term={{utm_term}}&utm_content={{utm_content}}&utm_campaign={{utm_campaign}}
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:17
I am testing the implementation of Tuktu. At this moment I don't see data on any of the Tuktu servers. What is the issue?
Erik Tromp
@ErikTromp
Jul 22 2016 08:18
what I just mentioned, the URL is not valid
so the pixel flow crashed
and needs to be rebooted
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:18
That is not the only request that has been send
Erik Tromp
@ErikTromp
Jul 22 2016 08:18
makes sense, but this is the one that killed it
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:18
Ok.
So it crashes any time a bad request is sent
Can we fix that?
Erik Tromp
@ErikTromp
Jul 22 2016 08:19
depending on what the request entails: yes
Tuktu follows the let-it-crash paradigm
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:19
Ok. I follow, the it always needs to be available paradigm;-)
Erik Tromp
@ErikTromp
Jul 22 2016 08:19
ie. if you misuse the platform, it's up to you as designer to cope with it
well Tuktu does not
fundamentally
I'll reboot the flow for you
they are running again
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:21
Anybody can crash the pixel-servers is the conclusion
Erik Tromp
@ErikTromp
Jul 22 2016 08:22
the conclusion is this can be dealt with easily, but it's not the default behavior of Tuktu
Tuktu can do anything, it's up to you to use it as desired
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:22
Great:-)
For now it is fine
Erik Tromp
@ErikTromp
Jul 22 2016 08:22
we can build in all sorts of checks everywhere for this instance of yours
but
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:23
Aparantly the load-balancer did not take the broken pixel-server out
Erik Tromp
@ErikTromp
Jul 22 2016 08:23
the whole philosophy of the let it crash paradigm is you cannot possibly enumerate ALL potential exceptions, so you will always miss one
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:23
Thats problem two
Erik Tromp
@ErikTromp
Jul 22 2016 08:23
I think that is because Tuktu will always continue to run, the flows however won't
saying that out loud; you should probably use a different URL for health checking
but there are 3 URLs you should check, so it's kind of hard to do anyway with AWS
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:24
Ok
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:34
blob
Is it intended that I don't see a second request? How else is the ga_id collected?
Erik Tromp
@ErikTromp
Jul 22 2016 08:37
I don't follow, there are 2 requests
one JS one pixel
seems okay to me
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:39
Ok. Ter bevestiging. Het is dus niet de bedoeling dat het stukje javascript dat in de response van tuktu zit ook daadwerkelijk wordt uitgevoerd?
blob
This message was deleted
Erik Tromp
@ErikTromp
Jul 22 2016 08:40
Not the subsequent XMLHTTPRequest no
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:40
blob
Erik Tromp
@ErikTromp
Jul 22 2016 08:40
since the image tag is inserted that does the magic here
Tuktu usually works differently, but this is what fits your design, from your screenshots it seems to work as expected
what happens is that the JS inserts an IMG tag to the DOM
that calls Tuktu.gif
that captures the data
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:42
Ok. I don't see any IMG-tag being inserted
it's there
3rd element in your log
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:42
I get it
Thanks!
Erik Tromp
@ErikTromp
Jul 22 2016 08:43
you're welcome :)
and as a primer: I'll make the let it crash optional
or at least: the let it crash by data :package:
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:43
:-)
Erik Tromp
@ErikTromp
Jul 22 2016 08:44
that should prevent malicious requests from terminating the flow
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:44
The next thing is that I still don't see any data appearing in the /var/www/html directory...
Erik Tromp
@ErikTromp
Jul 22 2016 08:44
yeah it's /var/www
there's some files there with timestamped trails
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:45
Very good
Ok. Checked that.
To update the nginx conf files, I can remove the servers, one by one, from the load-balancer. After the updates are ready I can re-attach them, I suppose
That would be the idea, right?
Erik Tromp
@ErikTromp
Jul 22 2016 08:49
yes that will work
as long as you keep at least 1 up
or within the load balancer pool
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:49
So, I removed pixel-server02 and pixel-server03
Right
Erik Tromp
@ErikTromp
Jul 22 2016 08:49
yeah
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:56
Why did you replace the ssl-certificates? We have paid ones... How can I be sure that they always refresh in time?
Erik Tromp
@ErikTromp
Jul 22 2016 08:57
  1. there is a script that does that
  2. I copied over your configuration of node1, it had multiple cert uses in there, that is what I meant in the e-mail I sent a while ago about the SSL problems
ie. there is no rationale for it, you can change it to using your root cert
ObjectivePartners
@ObjectivePartners
Jul 22 2016 08:58
I can, and I already did.
Ok
I will replace them once more then
We had some misunderstandments
Erik Tromp
@ErikTromp
Jul 22 2016 09:03
Sure
ObjectivePartners
@ObjectivePartners
Jul 22 2016 09:38
The certificate on pixel-server02 is not working. I guess the script is not installed properly. I will need to replace your certificate with my certificate...
Nginx says:

* Reloading nginx configuration nginx [fail]
root@pixel-server02:/etc/nginx/sites-enabled# nginx -t
nginx: [emerg] BIO_new_file("/etc/letsencrypt/live/pixel-server02.objectivetracking.com/fullchain.pem") failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/etc/letsencrypt/live/pixel-server02.objectivetracking.com/fullchain.pem','r') error:2006D080:BIO routines:BIO_new_file:no such file)
nginx: configuration file /etc/nginx/nginx.conf test failed
ObjectivePartners
@ObjectivePartners
Jul 22 2016 10:27
Ok. The certificates are done.
Next issue. I get a console.log warning...
blob
Chrome says:
js?customer=LEI&brand=BEL&domain=NL:7 Failed to execute 'write' on 'Document': It isn't possible to write into a document from an asynchronously-loaded external script unless it is explicitly opened.
It suggests that our js-tag is broken...
ObjectivePartners
@ObjectivePartners
Jul 22 2016 10:44
Ok. Since the js-tag is broken and our customer is waiting, I'm proceeding with the fix...
<script type="text/javascript">
(function() {
var objectivescript = document.createElement('script');
objectivescript.async = true;
objectivescript.type = 'text/javascript';
objectivescript.src = '//track01.objectiveportal.com/Tuktu.js/js?customer=LEI&brand=BV&domain=NL';
document.head.appendChild(objectivescript);
})();
</script>

Erik Tromp
@ErikTromp
Jul 22 2016 10:45
Its not the js tag
Its the img tag
ObjectivePartners
@ObjectivePartners
Jul 22 2016 10:46
Ok
Is it an issue?
Erik Tromp
@ErikTromp
Jul 22 2016 10:46
That the js tag inserts
Well I havent seen this before, should chevk if the ga img is loaded
ObjectivePartners
@ObjectivePartners
Jul 22 2016 10:47
This fix indeed does not work
Erik Tromp
@ErikTromp
Jul 22 2016 10:47
Im on the road right now tho
ObjectivePartners
@ObjectivePartners
Jul 22 2016 10:47
Ok
It gives a 200 response...
Erik Tromp
@ErikTromp
Jul 22 2016 10:53
The ga pixel that is loaded via JS too?
If so, then it will worl
ObjectivePartners
@ObjectivePartners
Jul 22 2016 10:56
I guess it does work. The warning does not look pretty though...
I read the stackexchange thread properly now:-P Tuktu.js should use another method than document.write to append the pixel.
It says:
You will need to replace any document.write() statements in that script with explicit DOM manipulations by creating the DOM elements and then inserting them into a particular parent with .appendChild() or .insertBefore() or setting .innerHTML or some mechanism for direct DOM manipulation like that.
Erik Tromp
@ErikTromp
Jul 22 2016 10:57
Yeah read it too
But cannot do anything right now
ObjectivePartners
@ObjectivePartners
Jul 22 2016 10:57
I understand
I need to communicate something though
I will say that we will fix it
Erik Tromp
@ErikTromp
Jul 22 2016 10:58
Sure
ObjectivePartners
@ObjectivePartners
Jul 22 2016 12:09
Is the problem with the disallowed characters solved? I see that NSI placed some broken banner tags. I hope this doesn't mean we can't measure LEI...
ObjectivePartners
@ObjectivePartners
Jul 22 2016 12:33
We might want to adjust the servertime on the servers...
Can I just do that?
I get:
root@pixel-server01:[/var/www]: hwclock --show
Fri 22 Jul 2016 12:32:28 PM UTC -0.885941 seconds
LEI heeft de js-tag op alle domeinen gezet... Ik zie nog geen data...
Hier is de lijst:

• Customer: LEI / Brand: TT / Domain: NL

• Customer: LEI / Brand: AR / Domain: NL
• Customer: LEI / Brand: AR / Domain: BE
• Customer: LEI / Brand: AR / Domain: FR

• Customer: LEI / Brand: BV / Domain: NL
• Customer: LEI / Brand: BV / Domain: BE
• Customer: LEI / Brand: BV / Domain: DE
• Customer: LEI / Brand: BV / Domain: FR
• Customer: LEI / Brand: BV / Domain: ES
• Customer: LEI / Brand: BV / Domain: IT
• Customer: LEI / Brand: BV / Domain: PL
• Customer: LEI / Brand: BV / Domain: COM

NTPape
@NTPape
Jul 22 2016 12:35
The clock seems to be about right, why?
ObjectivePartners
@ObjectivePartners
Jul 22 2016 12:36
The notation is different than usual in our timezone... This leads to confusion
ObjectivePartners
@ObjectivePartners
Jul 22 2016 14:14
Ok. I waited for more than an hour and I don't see any data for any of the LEI js-tags...
ObjectivePartners
@ObjectivePartners
Jul 22 2016 14:54
I see the data being logged in files beginning with LEI_BEL_NL... :-)
Erik Tromp
@ErikTromp
Jul 22 2016 14:55
yeah everything is written to a single file now, I will change the flow once I also add the non-let it crash
ObjectivePartners
@ObjectivePartners
Jul 22 2016 15:23
Great:-)