Communication between nodejs applications on a private IPv6 network

0

Hello, I have the following scenario: Dualstack VPC with two public subnets and two private subnets I don't have public IPv4, only private. Private subnets have an egress-only internet gateway Instances are configured with primary IPv6 In instance 1 I have an NdeJs application that needs to communicate (back and forth) with a NodeJs application in instance 2, but all my instances were frustrated. NodeJs applications are served via the HTTP protocol (NodeJs native server) and I have the impression that this is the problem, since IPv6 instances on a private network do not have incoming connections, as the gateway is output only. Could this be the problem? How can I solve it?

12 Answers
3

I'm not sure if I'm understanding this right, but if you have two EC2 instances -- instance 1 and instance 2 -- in the same VPC, they'll be able to connect to each other with private IPv4 addresses or their IPv6 addresses. The security group attached to instance 1 needs to allow outbound traffic to the security group of instance 2, and the security group of instance 2 must allow inbound traffic from the security group of instance 1. If you need instance 2 also to be able to connect to instance 1, you'll need the same setup the other way around, too.

Internet connectivity and public IP addressing aren't relevant, when you're only connecting inside your VPC.

If I didn't understand correctly, would you clarify to which address you're trying to connect when the connection fails. If the connection is attempted to a DNS name, then does that name point to the private IPv4 address or to the IPv6 address of either instance 1 or instance 2? And are both instances in the same VPC or in different VPCs?

EXPERT
Leo K
answered a month ago
profile picture
EXPERT
reviewed a month ago
profile pictureAWS
EXPERT
iBehr
reviewed a month ago
  • As mentioned, communication within a VPC does not require Internet or NAT Gateways. Focus on the Security Groups. If you have custom NACLs, check them as well (the default NACL would not block this traffic).

  • Thanks for the answer, you understood correctly. The instances are in the same VPC and the same private subnet and I connect the two with private IPv4. However, the connection does not work in NodeJs. The call from instance 1 to instance 2 always returns timeout.

1

If you are definitely connecting to the correct IP address and port number, then there are just two places where the connection could be blocked.

First, the security group attached to instance 1 must have two rules:

  1. inbound (ingress) rule that permits the required port number from the security group ID attached to instance 2
  2. outbound (egress) rule that permits the required port number to the security group ID attached to instance 2

The security group attached to instance 2 must have two similar rules:

  1. inbound (ingress) rule that permits the required port number from the security group ID attached to instance 1
  2. outbound (egress) rule that permits the required port number to the security group ID attached to instance 1

This way, instance 1 can connect to instance 2, and instance 2 can connect to instance 1.

Secondly, if you have a local firewall on the compute platform, such as "iptables" on Linux, it must also be configured to permit the connections.

EXPERT
Leo K
answered a month ago
EXPERT
reviewed a month ago
1

Hello everyone and thanks for the responses. I checked the security groups and they look fine. I tested the connection between the two instances using the CURL and NC commands and got a positive response

Example: From instance 1 to instance 2 curl -v http://instance-2_ipv4:app_port_instance-2 -> success! nc -zv instance-2_ipv4 app_port_instance-2 -> success!

From instance 2 to instance 1 curl -v http://instance-1_ipv4:app_port_instance-1 -> success! nc -zv instance-1_ipv4 app_port_instance-1 -> success!

But when the NodeJs application of instance 1 calls the application of instance 2, after a long time, I receive the timeout message

answered a month ago
EXPERT
reviewed a month ago
1

Okay, security groups and the internal operating system configurations are fine then.

To which address are you connecting from your Node.js application? Are you connecting by internal IPv4 address, IPv6 address, or DNS name? If you're using a DNS name, is it an AWS-owned DNS name (like ec2.internal or amazonaws.com) or your own domain name? Does the DNS name have an A (IPv4) record, AAAA (IPv6) record, or both, and does it/do they point to the IPv4, IPv6, or both addresses of the nodes?

EXPERT
Leo K
answered a month ago
1

I guess 3006 is the same port you tried successfully with curl. Can you use curl to test calling the same API/script that the Node.js application is trying to invoke? It's possible that network connections are working fine but that the API/script/function you're calling is just taking long (or forever) to respond, causing the Node.js application calling it to time out eventually.

EXPERT
Leo K
answered a month ago
1

The IPv4 VPC gateway endpoint you probably have in your VPC allows you to communicate with buckets in the local region with just your private IPs. However, a VPC is a regional construct and so are VPC endpoints, so you won't be able to reach buckets in remote regions through it. That's why curl isn't receiving a response from the remote region, while it would work for a public IPv4 of S3 in the local region (when routed via a VPC gateway endpoint in your VPC). By default, you'd connect to S3 in other regions over the public internet.

When your IPv6 default route points to an egress-only internet gateway or a regular internet gateway, you can connect to the IPv6 endpoints of S3 in remote regions. For us-east-1 (North Virginia), you should be able to reach this IPv6 address with curl: s3.dualstack.us-east-1.amazonaws.com. You won't need public IPv4 addresses to connect over IPv6.

EXPERT
Leo K
answered a month ago
0
Accepted Answer

Updating. It only works after I put the instance in a public subnet. In a private subnet, it doesn't work. Thanks everyone for the answers.

answered 6 days ago
0

I am connecting with NodeJs application as internal IPv4:

http://10.0.133.250:3006

Both instances are in the same private subnet in dual stack

answered a month ago
0

Yes, 3006 is the same port I used in CURL. I will do the suggested test. Thanks.

answered a month ago
0

Unfortunately I still haven't been able to resolve the problem. In fact, I have a lot of problems when using the IPv6 network on AWs. Since yesterday I can no longer connect via SSH, and it was working perfectly until Monday

answered a month ago
0

Hello everybody, I discovered that the instances I have in the dual stack VPC are unable to access AWS S3, and this is basically the problem with the timeout between applications. This same application works perfectly in the standard AWS VPC where I only have a public subnet. We are migrating to a dual stack VPC with IPv6 due to costs with the recent IPv4 charge. I made all the necessary changes to the javascript SDK to use dual stack. I added the recommended 'useDualstackEndpoint: true' property and also changed the direct call links to some bucket objects to use the dual stack link recommended by AWS. The VPC is in Ohio and the buket is in Northern Virginia, just like the old application, which works perfectly. I used the IP provided in the error stack and did a test, first on an instance that is in the default VPC and has a public IPv4 and I managed to get a positive response with the command below:

curl -v http://52.217.124.50

This same command from an instance that is in the dual stack VPC, which only has private IPv4 and IPv6, I get a timeout response, as below.

*   Trying 52.217.124.50:443...
* TCP_NODELAY set
* connect to 52.217.124.50 port 443 failed: Connection timed out
* Failed to connect to 52.217.124.50 port 443: Connection timed out
* Closing connection 0
curl: (28) Failed to connect to 52.217.124.50 port 443: Connection timed out

And this is the error in the NodeJs application.

Error: connect ETIMEDOUT at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1159:16) at TCPConnectWrap.callbackTrampoline (internal/async_hooks.js:130:17) {
  errno: -110,
  code: 'ETIMEDOUT',
  syscall: 'connect',
  address: '52.217.124.50',
  port: 443,
  name: 'TimeoutError',
 '$metadata': { attempts: 3, totalRetryDelay: 78
}

I'd be grateful for any help as I've been working on this for a long time and I need to finish it.

answered a month ago
0

Leo K, thanks for the answer. The application already uses the endpoint you suggested: s3.dualstack.us-east-1.amazonaws.com. This is currently the configuration I use. The VPC is in us-east-2 and the S3 bucket is in us-east-1. Could this be the problem? I don't understand why even using the correct endpoint the error stack always returns an IPv4 address.

Node app

const client = new S3({
    region: 'us-east-1',
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    useDualstackEndpoint: true
  });

Error:

Error: connect ETIMEDOUT 3.5.128.68:443
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1159:16)
at TCPConnectWrap.callbackTrampoline (internal/async_hooks.js:130:17) {
errno: -110,
code: 'ETIMEDOUT',
syscall: 'connect',
address: '3.5.128.68',
port: 443,
name: 'TimeoutError',
'$metadata': { attempts: 3, totalRetryDelay: 172 }
}

curl -v https://s3.dualstack.us-east-1.amazonaws.com

*   Trying 2600:1fa0:80fb:e6d0:34d9:c850:::443...
* TCP_NODELAY set
* Connected to s3.dualstack.us-east-1.amazonaws.com (2600:1fa0:80fb:e6d0:34d9:c850::) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: CN=s3.amazonaws.com
*  start date: May 25 00:00:00 2024 GMT
*  expire date: May  2 23:59:59 2025 GMT
*  subjectAltName: host "s3.dualstack.us-east-1.amazonaws.com" matched cert's "s3.dualstack.us-east-1.amazonaws.com"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M01
*  SSL certificate verify ok.
> GET / HTTP/1.1
> Host: s3.dualstack.us-east-1.amazonaws.com
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 307 Temporary Redirect
< x-amz-id-2: T4I/D0slF9bhix1cT6mmx4eQFhvmyUOHlEcBFhF3Jk4tDCm38Fda2uFPLKRg4DIsLslz4wTKiec=
< x-amz-request-id: K3KB71TJY8DF1V0X
< Date: Wed, 17 Jul 2024 20:33:12 GMT
< Location: https://aws.amazon.com/s3/
< Server: AmazonS3
< Content-Length: 0
<
* Connection #0 to host s3.dualstack.us-east-1.amazonaws.com left intact
answered a month ago