Objective

Some users have to route outbound traffic to external services (outside the company network) via forward proxy, because of security policies. To do that, they use the HTTP CONNECT method, but it requires configuring their applications to use those proxies and they would like to make them transparent for apps.

The expected solution is to make it possible to enable TCP tunneling in egress gateway. Then apps would perform standard requests that would be routed to the egress gateway, which would be responsible for tunneling received requests.

The diagrams below show the current and expected solutions.

Design Ideas

Envoy

- name: tcp

typed_config:

"@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy

cluster: forward_proxy_cluster

tunneling_config:

hostname: example.com:443

The only limitation of tunneling_config is that it supports only static hostname for now.

This means that it wouldn’t make sense to enable tunneling when filter_chain_match.server_names contains:

multiple server names
a server name with a wildcard

because in these cases it’s not known which hostname to set in tunneling_config.

I already started a discussion with the Envoy community and maintainers are open to extending tunneling_config to automatically set the right hostname in TLS listeners. Then using multiple hostnames or hostnames with wildcards could be allowed, because Envoy would rely on SNI value provided in runtime by TLS inspector filter. But in case of plain TCP listeners, it would still be needed to set a static hostname.

Istio

Since Envoy allows tunneling by TcpPoxy filters, Istio could also support it in gateway servers that listen on TCP or TLS, because for these protocols TcpProxy filters are generated. On the other hand, support for tunneling in HTTP or HTTPS gateway servers shouldn’t be considered at all, because there is no practical use case for it.

To enable this solution in Istio, I suggest extending DestinationRule.TrafficPolicy, with tunneling configuration as follows:

message TrafficPolicy {
message TunnelingSettings {
enum TunnelingMethod {
CONNECT = 1;
POST = 2;
}

// specifies which HTTP method to use to request a tunnel
TunnelingMethod method = 1;
}

// optional field that enables tunneling traffic;
// important limitation:
// allowed to apply only to routes for TCP or TLS listeners
TunnelingSettings tunnel = 6;
}

The implementation of handling DestinationRule would apply proper tunneling_config based on specified TrafficPolicy.TunnelingSettings and Gateway.Server.Hosts.

Without extending Envoy with automatically set hostname in tunneling_config, this solution would have also the following limitations:

allowed to apply only to routes which explicitly specify hostnames, i.e. don’t use wildcards;
allowed to apply only to routes which specify only one hostname in sniHosts, otherwise it will be not possible to create proper tunnel_config in the underlying TcpProxy filter; to enable tunneling for multiple destinations (hostnames), it would be necessary to create as many matching rules containing only one hostname in a related VirtualService, as many destination hosts needed.

Then enabling tunneling for all routes with destination host external-forward-proxy.com would require to apply the following resource:

apiVersion: networking.istio.io/v1alpha3

kind: DestinationRule

metadata:

spec:

host: external-forward-proxy.com

trafficPolicy:

tunnel:

method: CONNECT

In a case where only certain routes need tunneling, then subsets might be used to apply different rules for the same destination.

apiVersion: networking.istio.io/v1alpha3

kind: DestinationRule

metadata:

host: external-forward-proxy.com
subsets:
- name: google

trafficPolicy:

tunnel:

method: CONNECT

Future

By default a port in the request-target of the CONNECT method would be 443 or a port of the listener. But there might be use cases where users need a custom port for a particular hostname to request a tunnel, so we could extend TunnelingSettings with a field port.

message TunnelingSettings {
enum TunnelingMethod {
CONNECT = 1;
POST = 2;
}

// specifies which HTTP method to use to tunnel traffic
TunnelingMethod method = 1;

// specifies a custom port used in the request-target of CONNECT request,
// i.e. CONNECT <hostname>:<port>;
// 0 means that port used by a listener will be used in the request-target
uint32 port = 2;
}

apiVersion: networking.istio.io/v1alpha3

kind: DestinationRule

metadata:

host: external-forward-proxy.com
subsets:
- name: tunnel-google

trafficPolicy:

tunnel:

method: CONNECT
- name: tunnel-wikipedia
trafficPolicy:
tunnel:
method: POST
port: 8443

Custom ports might be really useful in case of a gateway listening on terminated TLS. In such a case, TLS routes cannot be applied and TCP routes must be used instead, but TCP routes do not allow SNI-based routing. The workaround is to create as many gateway servers (on different ports) as many SNI routes need to be performed, but then server ports may not match the target ports, so setting them in a destination rule would fix the problem.

Example

apiVersion: networking.istio.io/v1alpha3

kind: ServiceEntry

metadata:

spec:

hosts:

- external-forward-proxy.company.net

location: MESH_EXTERNAL

ports:

- number: 443

protocol: TLS

resolution: DNS

---
apiVersion: networking.istio.io/v1alpha3

kind: ServiceEntry

metadata:

spec:

hosts:

- www.example.com

location: MESH_EXTERNAL

ports:

- number: 443

protocol: TLS

resolution: DNS

---

apiVersion: networking.istio.io/v1alpha3

kind: VirtualService

metadata:

spec:

hosts:

- www.example.com

gateways:

- istio-egressgateway

- mesh

tls:

- match:

- gateways:

- mesh

port: 443

sniHosts:

- www.example.com

route:

- destination:

host: istio-egressgateway.istio-system.svc.cluster.local

port:

number: 443

- match:

- gateways:

- istio-egressgateway

port: 443

sniHosts:

- www.example.com

route:

- destination:

host: external-forward-proxy.company.net

port:

number: 443

---

apiVersion: networking.istio.io/v1alpha3

kind: DestinationRule

metadata:

host: external-forward-proxy.company.net

trafficPolicy:

tunnel:

method: CONNECT

---

apiVersion: networking.istio.io/v1alpha3

kind: Gateway

metadata:

spec:

selector:

istio: egressgateway

servers:

- port:

number: 443

protocol: TLS

hosts:

- www.example.com

tls:

mode: PASSTHROUGH

Shared with Istio Community