Can't ssh to instance when assign two NICs

Question

I have created an instance with two NICs:

nic0 with shared VPC without Pubic IP

nic1 with default vpc with Pubic IP

During the VM creation I could see notification.

Firewalls setup is not available for multiple network interfaces.

After the VM created I could not access the machine using Public IP Address.

I tried with shared VPC where I was able to ssh and ping the VM from that VPC.

The only problem is when try to access via Public IP address I could not able to get any response.

Answer 1

As per the gcloud documentation,

The Compute Engine DHCP server programs a default route only on the primary network interface of the VM(nic0). If we want to connect to the secondary interface using an external IP, we have to set a default route on that network interface through serial console.

If you want to ssh into the machine using the public IP assigned to nic1, you will have to change the default listening interface(which is by default nic0).

1. ssh into the machine using serial console

gcloud compute --project=your_project connect-to-serial-port vm_name --zone=your_zone

2. delete the default interface

ip route del default

3. add the new interface as default

ip route add default required_ip dev required_interface

You will find your required ip and required interface by ssh(ing) into the vm using a jump server and running the command ip route.

Important: Only use serial console and not any jump server to change the default route because if you will do this through a jump server, you will loose connection from the machine once you delete the default interface.

If ssh in serial console asks for a username and password and if you haven't set that, then you need to ssh into the vm using a jump server, run sudo passwd $USER , complete the process and then try the above mentioned method to ssh through nic1.

Jump server refers to that machine which is present in the same network as my private machine (not having any external IP), such that first we can ssh into jump server and from there, ssh(jump) into the private vm.

I know the answer is a bit late but if you still need any help, comment it out.

Answer 2

I get the same behavior when I reproduce your configuration.

I can reach the instance via the public IP if I revert the nics configurations.
I set nic0 to the default VPC with the public IP and nic1 to the shared VPC with no public IP.

If you wish to connect to an external IP on the secondary interface, you need to add routes. There is an explanation on Google cloud Platform documentation.

Also this document is a great source of information for your case study. I hope this helps.

Answer 3

If you can connect via the first IP, but not the second one, apparently you didn't change the routing table on the VM to make it work. I can't say it's clearly documented. This is mentioned in the troubleshooting section and here.

One way is to add to the VM the following startup script:

instance=http://metadata.google.internal/computeMetadata/v1/instance
get() { curl -sS "$instance/network-interfaces/1/$1" \
             -H 'Metadata-Flavor: Google'; }
ip=`get ip`
gw=`get gateway`
ip route add default via "$gw" table 1
ip rule add from "$ip" table 1

This makes packets with the source IP equal to the second IP go to the second gateway (not the first one, as it would be by default). For that I create a rule for such packets to use the routing table 1, where I add the second default route to the second gateway.

They also suggest to create a "to" rule, and a route for the second gateway, but that doesn't seem to be needed.

A terraform project that demonstrates this can be found here.