Thanks for Nothing, GPT-4!

Based on a true story.

Photo by Annie Spratt on Unsplash

I’m not here to tell you how great GPT-4 is. I have been using it as an assistant to many different tasks and am definitely satisfied with its performance.

However, I have also learned that it is not wise to take whatever GPT-4 tells you as granted without any tests or justification.

I’m here to tell you a short story that made me think this way.

Yes, I have seen many posts that proved how dumb GPT-4 can be. But, those are somewhat tricky questions. What I’m about to tell you can definitely be a real-life use case that you might ask someone, or GPT-4, for feedback.

I’m writing a unit test and can’t decide which one to use out of two slightly different options.

# option 1
def test_query(submission):
    query = submission.query
    assert query.lower().count("where") == 1

# option 2
def test_query(submission):
    assert submission.query.lower().count("where") == 1

Let’s see what GPT-4 thinks.

First prompt:

Do you consider the following function as Pythonic or is there a way to improve it?

def test_query(submission):
    query = submission.query
    assert query.lower().count("where") == 1