Local template,ISO upload fails

[rajujith]

ISSUE TYPE

• Bug Report

COMPONENT NAME

SSVM

CLOUDSTACK VERSION

4.18.0.0

CONFIGURATION

CloudStack with SSL/TLS enabled for System VMs for browser-based template/ISO upload.

OS / ENVIRONMENT

SUMMARY

Upload a template or ISO using the local upload / browser based upload method. It fails with the below message on the browser although the file upload progress shows 100% :
"Failed to upload ISO. - Error: Request failed with status code 502"

STEPS TO REPRODUCE

1. Navigate to Images, Templates/ ISO
2. Use use upload template from Local or upload ISO from Local
3. Fill in the rest of the details as appropriate and initiate the upload.

EXPECTED RESULTS

Template/ISO upload should work fine. 

ACTUAL RESULTS

Template/ISO file uploads fails

In the SSVM cloud.log we can see this error:

2023-04-17 13:21:36,229 INFO  [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-3:null) HEADER: expires=2023-04-17T14:21:35.215Z
2023-04-17 13:21:36,229 INFO  [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-3:null) HEADER: hostname=null
2023-04-17 13:21:36,230 INFO  [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-3:null) HEADER: content-length=343019751
2023-04-17 13:21:36,230 INFO  [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-3:null) URI: uuid=eab106be-023d-46cd-98c2-30cc3c6e167d
2023-04-17 13:21:36,232 ERROR [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-3:null) post request validation failed
com.cloud.exception.InvalidParameterValueException: signature validation failed.
	at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.validatePostUploadRequest(NfsSecondaryStorageResource.java:3526)
	at org.apache.cloudstack.storage.resource.HttpUploadServerHandler.channelRead0(HttpUploadServerHandler.java:163)
	at org.apache.cloudstack.storage.resource.HttpUploadServerHandler.channelRead0(HttpUploadServerHandler.java:65)

In the error.log

pass request body failed to 127.0.0.1:8210 (127.0.0.1)

The hostname is not passed properly resulting in the pre-validation of the signature failing.

[Rudio-Engineering]

I am also seeing this issue.

[levindecaro]

There are new issue on uploading local template and ISO in 4.18.1.

2023-09-19 23:19:24,993 INFO  [handler.logging.LoggingHandler] (nioEventLoopGroup-2-1:null) [id: 0x777c7e67, L:/0.0.0.0:8210] READ COMPLETE
2023-09-19 23:19:25,023 INFO  [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:null) HEADER: SIGNATURE=u0p7CCNWJ4Fiz2aLh0D/0qhJYDs=
2023-09-19 23:19:25,025 INFO  [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:null) HEADER: METADATA=XLapWnsiqHmOvDOFkrveFbw08pWeGo0Y8KyuGVRWH69pBY9a3LaO8KfDx9aDfBj2TjlmJdHJWusgi2Zet5TWiuCtnYQPBGprXL0a7P+7prgfxXyXhouwOjpJRmhD8vpxkGc9fVE55pIyvLmKyJbjOc6gntIqABXIEUg+e8Glr00YVf0gkR4b7+UW3LcQwXDDfowfjEAXFlotFmnqdje+EBbUazB+ABly2wW9UCItNjX+7LYsbXrSovF0qVUIgcUx1LNJ2kWTW31hmpUIUa4fPczKLknnu+JFMJoI1Ec7Y/oixVoigqWcTXL+tUJAH4QPFZHFCqTPKykiy0qQqujonpmrof7DbnvaSvH+EIGz+NG+I7XOAkLVVBrhIvknuPfWCvDxb5hAymQvzxk6dtB2ZyIhm4nP/v3VYSbQFIp+GxEwdoM7aUfc8ovDnmPWukafZuvOVZG5NN9oEmeA56W4pEtsKZ7+VBv5AbyCCp9TsltVRVZPKyj5z+4lmhZGO8RDN2ZOUM2jhfXgQSoobqDMifLwfywU6xxKjSN7XQaG00Cl52aNYe+kuHhwEoGfTOqWmEm6uzkVuFiiBQgHPNdmHkzx9cKlSRCeqOmU8Mx+TCVXZgwEr3RiITfIg1ZP3hmIi6RoVX6Id8q1VreQow==
2023-09-19 23:19:25,026 INFO  [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:null) HEADER: EXPIRES=2023-09-20T00:19:06.336Z
2023-09-19 23:19:25,026 INFO  [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:null) HEADER: HOST=ssvm.lab.example.com
2023-09-19 23:19:25,027 INFO  [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:null) HEADER: CONTENT_LENGTH=725221594
2023-09-19 23:19:25,028 INFO  [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:null) URI: uuid=43d7be4b-6c2c-4619-90c5-2b5021c0fc17
2023-09-19 23:19:25,037 INFO  [storage.resource.NfsSecondaryStorageResource] (nioEventLoopGroup-3-2:null) Determined host nfs.lab.example.com corresponds to IP 10.10.10.90
2023-09-19 23:19:25,055 ERROR [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:null) post request validation failed
com.cloud.exception.InvalidParameterValueException: The one time post url is already used and the upload is in ERROR state.
	at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createUploadEntity(NfsSecondaryStorageResource.java:3267)
	at org.apache.cloudstack.storage.resource.HttpUploadServerHandler.channelRead0(HttpUploadServerHandler.java:168)
	at org.apache.cloudstack.storage.resource.HttpUploadServerHandler.channelRead0(HttpUploadServerHandler.java:65)
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
	at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:328)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:315)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:429)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514)
	at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:829)

[Rudio-Engineering]

This was an issue in 4.18.0. I thought I saw where it had been fixed prior to this release but I guess it didn’t make it.

…

________________________________ From: levindecaro ***@***.***> Sent: Tuesday, September 19, 2023 7:29:53 PM To: apache/cloudstack ***@***.***> Cc: Marty Godsey ***@***.***>; Comment ***@***.***> Subject: Re: [apache/cloudstack] Local template,ISO upload fails (Issue #7442) WARNING: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. There are new issue on uploading local template and ISO in 4.18.1. 2023-09-19 23:19:24,993 INFO [handler.logging.LoggingHandler] (nioEventLoopGroup-2-1:null) [id: 0x777c7e67, L:/0.0.0.0:8210] READ COMPLETE 2023-09-19 23:19:25,023 INFO [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:null) HEADER: SIGNATURE=u0p7CCNWJ4Fiz2aLh0D/0qhJYDs= 2023-09-19 23:19:25,025 INFO [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:null) HEADER: METADATA=XLapWnsiqHmOvDOFkrveFbw08pWeGo0Y8KyuGVRWH69pBY9a3LaO8KfDx9aDfBj2TjlmJdHJWusgi2Zet5TWiuCtnYQPBGprXL0a7P+7prgfxXyXhouwOjpJRmhD8vpxkGc9fVE55pIyvLmKyJbjOc6gntIqABXIEUg+e8Glr00YVf0gkR4b7+UW3LcQwXDDfowfjEAXFlotFmnqdje+EBbUazB+ABly2wW9UCItNjX+7LYsbXrSovF0qVUIgcUx1LNJ2kWTW31hmpUIUa4fPczKLknnu+JFMJoI1Ec7Y/oixVoigqWcTXL+tUJAH4QPFZHFCqTPKykiy0qQqujonpmrof7DbnvaSvH+EIGz+NG+I7XOAkLVVBrhIvknuPfWCvDxb5hAymQvzxk6dtB2ZyIhm4nP/v3VYSbQFIp+GxEwdoM7aUfc8ovDnmPWukafZuvOVZG5NN9oEmeA56W4pEtsKZ7+VBv5AbyCCp9TsltVRVZPKyj5z+4lmhZGO8RDN2ZOUM2jhfXgQSoobqDMifLwfywU6xxKjSN7XQaG00Cl52aNYe+kuHhwEoGfTOqWmEm6uzkVuFiiBQgHPNdmHkzx9cKlSRCeqOmU8Mx+TCVXZgwEr3RiITfIg1ZP3hmIi6RoVX6Id8q1VreQow== 2023-09-19 23:19:25,026 INFO [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:null) HEADER: EXPIRES=2023-09-20T00:19:06.336Z 2023-09-19 23:19:25,026 INFO [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:null) HEADER: HOST=ssvm.lab.example.com 2023-09-19 23:19:25,027 INFO [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:null) HEADER: CONTENT_LENGTH=725221594 2023-09-19 23:19:25,028 INFO [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:null) URI: uuid=43d7be4b-6c2c-4619-90c5-2b5021c0fc17 2023-09-19 23:19:25,037 INFO [storage.resource.NfsSecondaryStorageResource] (nioEventLoopGroup-3-2:null) Determined host nfs.lab.example.com corresponds to IP 10.10.10.90 2023-09-19 23:19:25,055 ERROR [storage.resource.HttpUploadServerHandler] (nioEventLoopGroup-3-2:null) post request validation failed com.cloud.exception.InvalidParameterValueException: The one time post url is already used and the upload is in ERROR state. at org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource.createUploadEntity(NfsSecondaryStorageResource.java:3267) at org.apache.cloudstack.storage.resource.HttpUploadServerHandler.channelRead0(HttpUploadServerHandler.java:168) at org.apache.cloudstack.storage.resource.HttpUploadServerHandler.channelRead0(HttpUploadServerHandler.java:65) at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:328) at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:315) at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:429) at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:283) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931) at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700) at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635) at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552) at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514) at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1044) at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:829) — Reply to this email directly, view it on GitHub<#7442 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/A5SX7GOHK5DVQZB5GT3ALXLX3ITHDANCNFSM6AAAAAAXCC74DA>. You are receiving this because you commented.Message ID: ***@***.***>

[levindecaro]

right, i'd created a new #7981 for it anyway. thank you.