How to Mock a Rest API in Python

A few posts ago, we published a blog about how to use the Jira API. We did not write any unit tests for the application we wrote and that is exactly what we are going to do now. More specifically, we will focus on how we can unit test a Rest API.

1. Why Unit Tests Anyway?

Our main focus when writing software is building new features, fixing bugs. Of course, we need to test what we built, but we get the most joyful moment when our new developed feature works. Next step is to write unit tests… But, we already know it is working, so why spend much effort in it? It is much more fun to start with the next feature, right? So, we skip writing unit tests. But what happens when we need to extend the feature we wrote with some new functionality? How are we going to ensure we did not break the original feature? What if we want to refactor the code we have written? These are the moments we are glad when we notice someone took the effort to write unit tests. Unit tests will give us the confidence not breaking anything when changing the code (assuming the unit tests are well written and provide enough code coverage). It is therefore also common practice to run the unit tests as part of your CI/CD pipeline. When you do not like writing unit tests afterwards, you can also consider writing your unit tests first, let them fail and when you implement your code, the unit tests will pass (TDD: Test Driven Development). This way, when you have reached your uppermost moment of joy (It works!), you also have your unit tests passed (All the unit tests passed!). That doubles your moment of joy 🙂 .

2. Create Your First Unit Test

We will build upon the sources of the Jira time report generator. We are using Python 3.7 and PyCharm as IDE. First, let’s create a test directory and right-click the directory in PyCharm. Choose New - Python File and Python unit test. This creates the following default file:

import unittest

class MyTestCase(unittest.TestCase):
    def test_something(self):
        self.assertEqual(True, False)


if __name__ == '__main__':
    unittest.main()

Running this unit test obviously fails (True does not equals False), but we do have set up the basics for writing our own unit tests now.

3. Mocking a Rest API

We want to unit test the get_updated_issues function and this provides us a first challenge: the get_updated_issues function contains a call to the Jira Rest API. We do not want our unit test to be dependent of a third party service and therefore we need a way to mock the Rest API. There are several options to mock a Rest API, but we will make use of the requests-mock Python library which fits our needs.

Install the requests-mock Python library:

pip install requests_mock

3.1 Test Single Page Response

The get_updated_issues function will request the issues which are updated in a certain time period. In our unit test, we will verify the behavior when one page with results is retrieved (the Rest API supports pagination, but that is something for a next unit test):

def test_get_updated_issues_one_page(self):
    with open("issues_one_page.json", "r") as issues_file:
        mock_response = issues_file.read()

    expected_result = [
        {'expand': 'operations,versionedRepresentations,editmeta,changelog,renderedFields', 'id': '10005',
         'self': 'https://jira_url/rest/api/2/issue/10005', 'key': 'MYB-5'},
        {'expand': 'operations,versionedRepresentations,editmeta,changelog,renderedFields', 'id': '10004',
         'self': 'https://jira_url/rest/api/2/issue/10004', 'key': 'MYB-4'}]

    with requests_mock.Mocker() as m:
        m.register_uri('GET', '/rest/api/2/search', text=mock_response)
        response = jiratimereport.get_updated_issues("https://jira_url", "user_name", "api_token",  "MYB",
                                                     "2020-01-10", "2020-01-20", "")
    self.assertEqual(expected_result, response)

Let’s take a closer look at what is happening here. We have defined the Jira JSON response in file issues_one_page.json. At line 2 and 3, we read the contents of the file into variable mock_response. In line 5, we define the expected result with variable expected_result when the function get_updated_issues returns. At lines 11 up to 13 the magic happens. We register the URI we call from within the get_updated_issues function with the Mocker we defined. The third parameter of register_uri defines the response which should be returned from the mocked API call. At line 15 we call the get_updated_issues function and at the end we verify whether the response equals the expected result.

3.2 Test Paginated Response

The Jira API supports pagination. We added some functionality to the get_updated_issues function in order to handle paginated responses. The JSON response contains three fields for this:

  • startAt: indicates from which result the page should be retrieved;
  • maxResults: the number of maximum results which are returned within one response;
  • total: the total number of results.

The maxResults for retrieving issues is, at the time of writing, 50. If we would like to test this against a real Jira server, we would have to create at least 51 issues. But for testing with our unit test, we can easily change the response in order that field maxResults returns 2 for example, which makes it a lot easier to test. Our paginated unit test looks as follows:

def test_get_updated_issues_multiple_pages(self):
    with open("issues_multiple_first_page.json", "r") as issues_first_file:
        mock_response_first_page = issues_first_file.read()

    with open("issues_multiple_second_page.json", "r") as issues_second_file:
        mock_response_second_page = issues_second_file.read()

    expected_result = [
        {'expand': 'operations,versionedRepresentations,editmeta,changelog,renderedFields', 'id': '10005',
         'self': 'https://jira_url/rest/api/2/issue/10005', 'key': 'MYB-5'},
        {'expand': 'operations,versionedRepresentations,editmeta,changelog,renderedFields', 'id': '10004',
         'self': 'https://jira_url/rest/api/2/issue/10004', 'key': 'MYB-4'},
        {'expand': 'operations,versionedRepresentations,editmeta,changelog,renderedFields', 'id': '10006',
         'self': 'https://jira_url/rest/api/2/issue/10006', 'key': 'MYB-6'}]

    with requests_mock.Mocker() as m:
        m.register_uri('GET', '/rest/api/2/search', [{'text': mock_response_first_page},
                                                     {'text': mock_response_second_page}])
        response = jiratimereport.get_updated_issues("https://jira_url", "user_name", "api_token",  "MYB",
                                                     "2020-01-10", "2020-01-20", "")

    self.assertEqual(expected_result, response)

The unit test looks quite similar to the one for the single page response. We defined two mock responses this time, because the Jira API will be called twice and we want to return two different responses. The expected_result variable contains the combined result of both API calls. The real difference can be seen at line 17. Instead of defining a single mock response, we now define a list of mock responses. When the API is called more than the defined responses, the latest defined mock response is returned again.

3.3 Test Failed

The unit tests above all pass. But do they fail when something is wrong? We can therefore change e.g. the following line in the get_updated_issues function:

issues_json.extend(response_json['issues'])

Change it with:

issues_json = response_json['issues']

This will ensure that only the response of the first API call will be added to the returned issues list, but not the response of succeeding API calls. Run both tests again. The test_get_updated_issues_one_page passes, but the test_get_updated_issues_multiple_pages fails:

AssertionError: Lists differ

3.4 Test Multiple URI’s

The Jira work logs are to be retrieved per issue. We therefore need to register more than one URI with different responses for each issue. The response of the get_work_logs function returns a list of WorkLog objects which we can assert.

def test_get_work_logs_one_page(self):
    with open("work_logs_first_issue_one_page.json", "r") as first_issue_file:
        mock_response_first_issue = first_issue_file.read()

    with open("work_logs_second_issue_one_page.json", "r") as second_issue_file:
        mock_response_second_issue = second_issue_file.read()

    issues_json = [
        {'expand': 'operations,versionedRepresentations,editmeta,changelog,renderedFields', 'id': '10005',
         'self': 'https://jira_url/rest/api/2/issue/10005', 'key': 'MYB-5'},
        {'expand': 'operations,versionedRepresentations,editmeta,changelog,renderedFields', 'id': '10004',
         'self': 'https://jira_url/rest/api/2/issue/10004', 'key': 'MYB-4'}]

    with requests_mock.Mocker() as m:
        m.register_uri('GET', '/rest/api/2/issue/MYB-5/worklog/', text=mock_response_first_issue)
        m.register_uri('GET', '/rest/api/2/issue/MYB-4/worklog/', text=mock_response_second_issue)
        work_logs = jiratimereport.get_work_logs("https://jira_url", "user_name", "api_token",
                                                 "2020-01-10", "2020-01-20", "", issues_json)

    self.assertEqual(work_logs[0], WorkLog("MYB-5", datetime(2020, 1, 18), 3600, "John Doe"))
    self.assertEqual(work_logs[1], WorkLog("MYB-5", datetime(2020, 1, 18), 5400, "John Doe"))
    self.assertEqual(work_logs[2], WorkLog("MYB-4", datetime(2020, 1, 12), 3600, "John Doe"))

4. Conclusion

Writing unit tests is absolutely necessary when you want to develop software in a professional manner. In this post, we took a look at how to unit test a Rest API by means of the requests-mock Python library. We only scratched the surface of what this library has to offer, but our first impressions are very good.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.