Tuesday, April 16, 2024
HomeArtificial IntelligenceGetting the Proper Reply from ChatGPT – O’Reilly

Getting the Proper Reply from ChatGPT – O’Reilly


A few days in the past, I used to be fascinated with what you wanted to know to make use of ChatGPT (or Bing/Sydney, or any related service). It’s simple to ask it questions, however everyone knows that these giant language fashions incessantly generate false solutions. Which raises the query: If I ask ChatGPT one thing, how a lot do I must know to find out whether or not the reply is appropriate?

So I did a fast experiment. As a brief programming mission, quite a few years in the past I made an inventory of all of the prime numbers lower than 100 million. I used this listing to create a 16-digit quantity that was the product of two 8-digit primes (99999787 instances 99999821 is 9999960800038127). I then requested ChatGPT whether or not this quantity was prime, and the way it decided whether or not the quantity was prime.


Be taught sooner. Dig deeper. See farther.

ChatGPT accurately answered that this quantity was not prime. That is considerably stunning as a result of, when you’ve learn a lot about ChatGPT, you recognize that math isn’t one in every of its sturdy factors. (There’s in all probability a giant listing of prime numbers someplace in its coaching set.) Nonetheless, its reasoning was incorrect–and that’s much more fascinating. ChatGPT gave me a bunch of Python code that applied the Miller-Rabin primality take a look at, and stated that my quantity was divisible by 29. The code as given had a few primary syntactic errors–however that wasn’t the one downside. First, 9999960800038127 isn’t divisible by 29 (I’ll allow you to show this to your self). After fixing the plain errors, the Python code regarded like an accurate implementation of Miller-Rabin–however the quantity that Miller-Rabin outputs isn’t an element, it’s a “witness” that attests to the very fact the quantity you’re testing isn’t prime. The quantity it outputs additionally isn’t 29. So ChatGPT didn’t truly run this system; not stunning, many commentators have famous that ChatGPT doesn’t run the code that it writes. It additionally misunderstood what the algorithm does and what its output means, and that’s a extra critical error.

I then requested it to rethink the rationale for its earlier reply, and acquired a really well mannered apology for being incorrect, along with a distinct Python program. This program was appropriate from the beginning. It was a brute-force primality take a look at that attempted every integer (each odd and even!) smaller than the sq. root of the quantity below take a look at. Neither elegant nor performant, however appropriate. However once more, as a result of ChatGPT doesn’t truly run this system, it gave me a brand new listing of “prime components”–none of which had been appropriate. Apparently, it included its anticipated (and incorrect) output within the code:

      n = 9999960800038127
      components = factorize(n)
      print(components) # prints [193, 518401, 3215031751]

I’m not claiming that ChatGPT is ineffective–removed from it. It’s good at suggesting methods to unravel an issue, and may lead you to the fitting answer, whether or not or not it offers you an accurate reply. Miller-Rabin is fascinating; I knew it existed, however wouldn’t have bothered to look it up if I wasn’t prompted. (That’s a pleasant irony: I used to be successfully prompted by ChatGPT.)

Getting again to the unique query: ChatGPT is sweet at offering “solutions” to questions, but when it’s worthwhile to know that a solution is appropriate, it’s essential to both be able to fixing the issue your self, or doing the analysis you’d want to unravel that downside. That’s in all probability a win, however it’s important to be cautious. Don’t put ChatGPT in conditions the place correctness is a matter except you’re prepared and capable of do the arduous work your self.



RELATED ARTICLES

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Most Popular

Recent Comments