it appears you use a custom resource in cloudformation to trigger lambda to update the github hooks?
@mrjk05 it's not so much lambci pointing at other sources – it's more the other way around! At the moment it doesn't appear that Gitlab supports SNS, which is what LambCI uses as a Lambda hook to get notified of git updates: https://gitlab.com/gitlab-org/gitlab-ce/issues/3486
It would be possible to integrate with GitLab without SNS – using API Gateway directly, in front of Lambda – but this would need some work to be done to create that hook
hey @mhart - I've never been able to get a build to work. I keep getting
npm ERR! tar.unpack untar error /tmp/lambci/home/.npm/lodash/4.17.4/package.tgz
npm ERR! tar.unpack untar error /tmp/lambci/home/.npm/lodash/4.17.4/package.tgz
npm ERR! tar.unpack untar error /tmp/lambci/home/.npm/core-js/2.4.1/package.tgz Any ideas what's going on?
Lambda only has 500MB available – so if your deps use more than that with npm@2 then you're outta luck. I'm going to upgrade to using the node6.10 runtime at some point soon, which has npm3 on it by default – which should use less space when npm installing due to its deduping. You could try to use nave right now to achieve this too: https://github.com/lambci/lambci#nodejs
For the record, it seems the only way to get it working (anywhere) is using git+https a la
git+https://<token>:x-oauth-basic@github.com/<user>/<repo>.git. lambci would have to preprocess package.json and replace all github: or git+ssh...github entries with the access token version.
though... that only works for direct deps. hmmm.
I'd rather not hardcode my NPM_TOKEN in the build-args or the docker file
I opened an issue about it on the lambci repo, I think it's an easy fix.
feel free to ping me if I can help with it
Hi
yesterday lambci started to throw weird errors when using gcc. my build cmd looks like this:
yesterday lambci started to throw weird errors when using gcc. my build cmd looks like this:
. ~/init/gcc && npm install && npm test. output is$ . ~/init/gcc && npm install && npm run test
Installing GCC 4.8.5...
++ curl -sSL https://lambci.s3.amazonaws.com/binaries/gcc-4.8.5.tgz
++ tar -xz -C /tmp
++ set +x
GCC setup complete
/bin/bash: line 1: 33 Segmentation fault (core dumped) npm install
Build #1 failed: Command ". ~/init/gcc && npm install && npm run test" failed with code 139
if i remove
. ~/init/gcc part, then build will succeed
does anyone know whats wrong? :(
I updated the Docker image to expect those as parameters on runbuild.sh, and updated the Lambda function to pass those env vars in the dockerBuild function
I am hosting the revised image locally and redeployed the Lambda function. However ecs doesn't output any logs, so I might have messed up something. Wondering if there is an easier approach (other than hardcoding on lambci.json under the Docker configs)
I just want to deploy whatever I build and test to S3... can't do on Lambda only because hit the disk size limit
If you're using the aws-sdk for example, it will just get the credentials automatically
@interisti that seg fault error should be fixed now – it was due to updates on the Lambda system that they did without warning. Let me know if it works for you now or not
@mhart this is the Dockerfile.test in my repo
FROM mhart/alpine-node:6
RUN apk update && \
apk upgrade && \
apk add \
util-linux \
pciutils \
usbutils \
coreutils \
binutils \
findutils \
grep \
curl \
bash \
git
RUN apk add \
python \
py-pip \
&& \
pip install --upgrade awscli
WORKDIR /tmp/lambci/build
USER root
# Build node_modules
ADD package.json /buildserver/
RUN cd /buildserver && npm install
# Now add the rest, the above layers will be cached if package.json doesn't change
ADD . /tmp/lambci/build
# Move npm modules over
RUN rm -rf /tmp/lambci/build/server/node_modules && mv /buildserver/node_modules /tmp/lambci/build/server/
RUN cd /tmp/lambci/build && ./lambci.sh
and in lambci.sh I have a bash script that tests and builds the code. The final step if to call an aws s3 sync... which is why I need the credentials
It builds fine, the only problem is I cant deploy to s3 because I dont have access to the credentials. Not sure how to rely on the IAM role for that
When I ran this from Lambda (no Docker), it worked because the credentials are available as env variables. But I hit the 500mb limit. And in Docker the secret env vars are not passed
so you'll just need to give that role access to whatever s3 bucket you need
If you're using the CloudFormation template, you can just update the permissions from there – and update your stack: https://github.com/lambci/ecs/blob/master/cluster.template#L97-L137
@mhart I am trying to understand what controls the constraints of how many concurrent deployments are possible when using lambci/ecs.
On the CloudFormation template, the InstanceType parameter will determine the disk space/available computing power.
How about the AutoScalingGroup properties? Do the defaults mean that only 1 build can be run at the same time? If I change that do I also need to change the CreationPolicy.ResourceSignal?
On the CloudFormation template, the InstanceType parameter will determine the disk space/available computing power.
How about the AutoScalingGroup properties? Do the defaults mean that only 1 build can be run at the same time? If I change that do I also need to change the CreationPolicy.ResourceSignal?
Another thing is I have run into a "Thin pool" error, which went away once I cleaned the old Docker images. How do you deal with that, do you have a cron job on the ECS instance host that does the cleanup regularly?
I believe it is because I am building the Dockerfile.test everytime. I have seen that lambci/ecs runs
build followed by run, but it is not clear to me how to pass a shell script to the run stage. Maybe if I could leverage $LAMBCI_DOCKER_RUN_ARGS?
Having a bit of trouble just getting started with this. I attempted to use "launch stack" as well as a few different ways (all really used same template though) and I get an error on "Custom::ConfigUpdater" every time. It looks like at the point in time "Custom::ConfigUpdater" is run the ConfigTable is not created yet. I adjusted the template and added "DependsOn: ["ConfigTable"]" for "ConfigUpdater" but the result of this was a completely different error: " Failed to create resource. User: arn:aws:sts::837930595040:assumed-role/lambci-private-LambdaExecution-LR33W5LBSV98/lambci-private-build is not authorized to perform: iam:PassRole on resource: arn:aws:iam::837930595040:role/lambci-private-SnsFailures-1Q43HAMIVMMR5" . I am honestly not very familiar with CloudFormation and all the config options that seem to be available. Is nobody else having this issue with this stack?
N. Virginia