Sending JSON payload to a post request ?

[asmodehn]

Strangely this hasn't been working for me as simply as I expected...

I can't see an example that sends json data as part of the POST request in the README.
Also when I dive into the code I see only headers and params being taken care of in the request :

    def __init__(self, **kwargs):
        self.extra_request_opts = kwargs

    def request(self, api_url, method, headers, params, is_raw):
        self.logger.info(
            '\n Invoking REST Call... api_url: %s, method: %s, headers %s : ', api_url, method, headers)

        try:
            func = getattr(requests, method)
        except AttributeError:
            self.logger.error('undefined HTTP method!!! %s', method)
            raise
        response = func(api_url, headers=headers, params=params, **self.extra_request_opts)

I dont see either data or json as I would expect here...
Any hint ? Something I missed or something that needs fixing ?

[chitamoor]

You should be able to submit a POST request simply by setting the method type to POST. I believe the documentation has an example of a POST method -
Perform an HTTP POST

testSteps: [
{
"name":"Name of TestStep",
"apiUrl":"http://example/api/v1/helloworld/print",
"headers":{
...
},
"method":"post"
"params":{
"param_1":"value1",
"param_2":"value2"
},
....
}
]

I have used this extensively in the past to POST params to an endpoint, but if it doesn't work let me know.

[asmodehn]

Yes I tried this, and I m using flask om the backed. It seems these are url parameters, not the payload of the request...

[asmodehn]

To be more precise, from what I understand, with Rester I can currently do this :
http://docs.python-requests.org/en/latest/user/quickstart/#passing-parameters-in-urls
but I cannot do that :
http://docs.python-requests.org/en/latest/user/quickstart/#more-complicated-post-requests

[chitamoor]

You are absolutely right in that Rester currently supports only method one (params as POST), and I agree that method two (POST payload) would be useful. When I get sometime I will look into implementing this, but if you need this right away, feel free to send in a pull request.

I am thinking it would something like -

testSteps: [
{
"name":"Name of TestStep",
"apiUrl":"http://example/api/v1/helloworld/print",
"headers":{
},
"payLoad": {
}
},

[asmodehn]

I have been looking at it, but I must admit I am puzzled at the DictWrapper...
What is it used for and why is it useful ?

It does some strange thing with lists :

        if isinstance(d, list):  # if the first level element is a list
            setattr(self, "._length", len(d))
            for index, item in enumerate(d):
                setattr(self, "[%s]" % index, DictWrapper(item))

and seems used with everything in the TestCase loader :

    def load(self):
        with open(self.filename) as fh:
            data = load(self.filename, fh)
            self._load(data)

    def _load(self, data):
        self.data = DictWrapper(data)

I was expecting to just pass my payload dictionary ( it does contain lists ) from the jsonfile, into the python code as a simple dict, but it looks like I maybe have to change the testcase loader somehow ?

Thank for any info & advice.

[asmodehn]

I started a pull request so we can discuss there with code : #15

[chitamoor]

Yes, DictWrapper is a confusing piece of code. The reason it exists is that it lets me dynamically map a JSON payload into a nice (python) object graph syntax which I can directly manipulate. For e.g. I can refer to elements of JSON payload using expression like "parant[0].child[2].property". This is how I am able to do assert statements. I am surprised why python doesn't support this out of the box i.e. JSON to object graph. I can't wait to get rid of this code some day. :)

Anyways, I took a quick look at your code, which mostly looks good. I need to make one change and test it.

Basically replace the following
data = self._build_data_dict(test_step)

with
data = test_step.payload

I might need to dump the DictWrapper (payload object) into an actual JSON object that "requests" expects, but let me test it locally.

[asmodehn]

I think at that point the test_step.payload is already a DictWrapper that cannot be serialized :

            # process and set up params
            params = self._build_param_dict(test_step)

            # process and set up data
            data = getattr(test_step, 'payload', None) <rester.struct.DictWrapper object at 0x7fa63c0675d0>

[asmodehn]

Since I am not really sure of the behavior of the dictwrapper, I realized what was missing is unit tests for it.

So I started a simple unit test structure there #19
Let me know what you think about it.

Once we have a complete unit test for what is needed in DictWrapper now, we can think about improving it (https://jsonpickle.github.io/ ?) without any risk of breaking something else around.

[asmodehn]

I had a deeper look at the code, and I understand more now.
If you :

• don't want to have to do test.get('testSteps',[]).get('asserts',{}).get('payload',{}) which is not really nice,
• don't want to have an extra DictWrapper that doesn't implement all features that developers are used to, and is not directly related with the purpose of Rester

I believe the way to go is to implement your TestCase object in a way that can be constructed from serialized (pickled) data : https://docs.python.org/2/library/pickle.html

it just happens that our pickled format is json.
The trick is then to add a field to specify which type needs to be reconstructed when unpickling from python code.

That is actually what jsonpickle does:

>>> import jsonpickle
>>> class Thing(object):
...     def __init__(self, name):
...         self.name = name
... 
>>> obj = Thing('Awesome')
>>> frozen = jsonpickle.encode(obj)
>>> hot = jsonpickle.decode(frozen)
>>> type(hot)
<class '__main__.Thing'>
>>> frozen
'{"py/object": "__main__.Thing", "name": "Awesome"}'

[asmodehn]

I made an experiment to convert Rester to use jsonpickle instead of DictWrapper.
Here it is : #20
I made the changes only for one test.
Could be better but we can already have direct JSON -> python object conversion.
Anyway let me know what you think about it.