DefaultHttpClient is being used, which does not use any HTTP connection pooling? Any specific reason to not using HTTP connection pooling ? Does WinRM not support re-using connections? This is becoming a serious issue in our environment, if we attempt to launch 18 or more nodes, with in couple of minutes of launching, most of the nodes goes offline due to connection termination error... after analyzing the network transmission data using tcpdump, we found that due to too many short lived connections to WinRM, number of available ports is getting exhausted and ports in TIME_WAIT state getting re-used in Jenkins server, Windows just simply does not acknowledge requests when it detects a port re-use.. Has this issue been observed earlier? Is it worth replacing the DefaultHttpClient with a custom client with pooling http connection manager?
8 replies
Has anybody observed the behavior of the plugin when it's getting super slow (scales literally by 1 slave at a time) on launching EC2s with spot-block enabled when AWS has a shortage with spot availability? Usually, the launching of the slaves happens by big chunks when there is a huge queue of waiting for executor builds...
5 replies
@res0nance @slide @alok0310 - I have tried connecting to windows 2016 using the ssh method described above and copying the ssh key but I am still unable to connect. Can you please let me know if there are any other steps to be followed?
Steps followed :
- Created a Windows EC2 machine and run the poweshell script provided by @res0nance and made sure I am able to connect via ssh
- Created and AMI and configured it in the Jenkins to use Unix method
- Provisioned an agent but Jenkins is unable to connect
Note: I have tested the ssh connectivity of the new agent created by jenkins and I am able to login but Jenkins is unable to login
Error:
INFO: The instance EC2 (AWS-sandbox) - windows-2016 (i-05a178d76ba4XXXX) has a blank console. Maybe the console is yet not available. If enough time has passed, consider changing the key verification strategy or the AMI used by one printing out the host key in the instance console
Jun 02, 2020 6:20:15 PM hudson.plugins.ec2.EC2Cloud
INFO: The instance console is blank. Cannot check the key. The connection to EC2 (AWS-sandbox) - windows-2016 (i-05a178d76ba4XXXX) is not allowed
Jun 02, 2020 6:20:15 PM hudson.plugins.ec2.EC2Cloud
INFO: Failed to connect via ssh: There was a problem while connecting to XX.XX.XX.XX:22
Jun 02, 2020 6:20:15 PM hudson.plugins.ec2.EC2Cloud
INFO: Waiting for SSH to come up. Sleeping 5.
Jun 02, 2020 6:20:16 PM hudson.plugins.ec2.EC2Cloud
INFO: Connecting to XX.XX.XX.XX on port 22, with timeout 10000.
@slide - Yes port 22 is open. I am able to connect to the server using ssh directly.
I have tried changing the "Host Key Verification Strategy" to 'accept-new' and off but AWS is terminating the instances within few minutes. Maybe it is thinking as a "man-in-the-middle" attack since I am using public IP to connect.
Are you using public IP or private IP for connection?
I also want to test regular connection to Windows EC2 server using "winrm".
Does anyone have details on how to get it working? I applied all the settings as described in the plugin and created an image but it always hangs at "connecting to (XX.XX.XX.XX) with WinRM as Administrator" and never connects.
I have the same problem as @donepudi369 and @bsubbaraman. I tried to figured it out and I discovered that it is hanging when creating the SMBClient.
I added some logs to the code:
log.log(Level.FINE, "Inside WinConnection constructor");
this.host = host;
this.username = username;
this.password = password;
log.log(Level.FINE, "Creating SMBClient");
this.smbclient = new SMBClient();
log.log(Level.FINE, "SMBClient created");
I can see the Inside WinConnection constructor and Creating SMBClient but it seems like SMBClient() never returns. The "SMBClient created" is never logged.
Hi, i was tasked to get ec2 in windows working. After spending some hours trying winRM, switched to the openssh was. (i havent read this chat until now, btw on recent enough windows (win10 or win serv 2019) the SSH can be installed with more easily than said above)
Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0
Start-Service sshd
Set-Service -Name sshd -StartupType 'Automatic'
New-ItemProperty -Path "HKLM:\SOFTWARE\OpenSSH" -Name DefaultShell -Value "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe" -PropertyType String -Force
It hasn't appeared in my mind to run the plugin in linux mode so i just started a java IDE, and rework the EC2WindowsLauncher.java to use SSH (copypasta driven development from Ec2UnixLauncher.java)
It is on https://github.com/ultinous-dancsa/ec2-plugin/tree/windows-over-ssh-poc
(its code quality is somewhere between the spagetti and "i just want a working PoC within 3 hours with no prior knowledge")
It works with password auth, the ami only had the ssh, and java install and a user creation
Do you plan to implement something like this in the plugin, or i've just read above the unix setting works for windows too if ssh is installed?
The instance EC2 Amazon-Linux 2 (i-xxxxxxxxxxxxx) has a blank console. Maybe the console is yet not available. If enough time has passed, consider changing the key verification strategy or the AMI used by one printing out the host key in the instance console
...
INFO: The instance console is blank. Cannot check the key. The connection to EC2 Amazon-Linux 2 (i-xxxxxxxxxxxxx) is not allowed
...
HTTP ERROR 404 Not Found