Detection of failed services during routing

[pavel-jares-bcm]

There is a mechanism how to handle network issue during resending request to a service:

api-layer/gateway-service/src/main/java/org/zowe/apiml/gateway/config/NettyRoutingFilterApiml.java

Lines 85 to 94 in 606d2b3

	public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
	return super.filter(exchange, chain).onErrorResume(e -> {
	if (e.getCause() instanceof ConnectException) {
	var uri = exchange.getRequest().getURI();
	return Mono.error(new ServiceNotAccessibleException(String.format("Service is not available at %s://%s:%d", uri.getScheme(), uri.getHost(), uri.getPort()), e));
	}
	return Mono.error(e);
	});
	}

The aim is detect that service is down. In this case GW returns 503 which is ok. But the condition is too general and could include other communication issue. The issue is then log in exception handler:

api-layer/gateway-service/src/main/java/org/zowe/apiml/gateway/controllers/GatewayExceptionHandler.java

Lines 172 to 176 in 606d2b3

	@ExceptionHandler({ServiceNotAccessibleException.class, WebClientResponseException.ServiceUnavailable.class})
	public Mono<Void> handleServiceNotAccessibleException(ServerWebExchange exchange, Exception ex) {
	log.debug("A service is not available at the moment to finish request {}: {}", exchange.getRequest().getURI(), ex.getMessage());
	return setBodyResponse(exchange, SC_SERVICE_UNAVAILABLE, "org.zowe.apiml.common.serviceUnavailable", exchange.getRequest().getPath());
	}

It generates debug message like:

A service is not available at the moment to finish request https://<GW host>:<GW port>/<path>: Service is not available at https://<service host>:<service port>

It ignores the cause of the error. It is not helpful to understand the reason of failure.

The aim of this issue is:

• provide beter debug message
• decide what exception could be logged as 503 and what as 500
  • 503 should be only request where is missing the remote site
  • exception about certificate (PKIX, etc.) shouldn't end with 503