ChatGPT Isn’t as Good at Coding as We Thought::undefined

  • drkt@feddit.dk
    link
    fedilink
    English
    arrow-up
    59
    arrow-down
    2
    ·
    1 year ago

    Who are ‘we’? Surely not anyone who’s ever used it?

      • tenextrathrills@lemmynsfw.com
        link
        fedilink
        English
        arrow-up
        17
        arrow-down
        2
        ·
        1 year ago

        I would hate to see your code if you think ChatGPT’s code works great. I’m guessing you’ve never asked it do anything complex.

        • abhibeckert@lemmy.world
          link
          fedilink
          English
          arrow-up
          5
          arrow-down
          1
          ·
          edit-2
          1 year ago

          So this isn’t a real example, it’s just something I slapped together now as an example of how I generally use ChatGPT 4.0. In a more realistic scenario, I’d be asking more detailed questions, pasting in my existing code, and asking the AI to write smaller sections (maybe 10 lines of code at a time).

          A single chat might run for five or six hours, continuously discussing an individual task I’m working on, and I won’t just be asking it to write a bunch code. For example in this case I might spend 10 minutes hashing out the exact contents of the ‘Activity’ record (or, maybe I already did that a month ago, and I might just paste in an example). I’d also be testing the code as I go and writing some of it myself, occasionally asking questions like ‘how do I format a date as 4 Feb 2016’ in JavaScript?’ or ‘this line of code fails with X. Why?’

          In those five or six hours, I estimate I’m able to get two or three days of work done (at my level of productivity before I started paying $20/month for ChatGPT+).

          I’m also only pasting the code - ChatGPT also explains the code it outputs, and often those explanations are more useful than the actual code.

          In JavaScript write a “reserveSeat(‘A42’, ‘Adult’)” function that sends an ActivityStreams 2.0 activity to (this.apiUrl)/reserve and inform the user if it failed to be reserved

          class SeatReservationService {
            constructor(apiUrl) {
              this.apiUrl = apiUrl;
            }
          
            async reserveSeat(seatNumber, seatType) {
              // Construct an ActivityStreams 2.0 compliant JSON object
              const activity = {
                '@context': 'https://www.w3.org/ns/activitystreams',
                'type': 'Reserve',
                'actor': {
                  'type': 'Person',
                  'name': 'User'
                },
                'object': {
                  'type': 'Seat',
                  'id': seatNumber,
                  'category': seatType
                }
              };
          
              try {
                // Send the activity object to the specified API endpoint using an HTTP POST request
                const response = await fetch(`${this.apiUrl}/reserve`, {
                  method: 'POST',
                  headers: {
                    'Content-Type': 'application/json'
                  },
                  body: JSON.stringify(activity)
                });
          
                // Handle the server’s response
                if (response.ok) {
                  // If the request was successful, log a success message
                  console.log('Seat reserved successfully.');
                } else {
                  // If the server responds with a status code other than 2xx, throw an error
                  throw new Error('Failed to reserve seat. Server responded with ' + response.status);
                }
              } catch (error) {
                // Inform the user if the reservation failed
                console.error('Failed to reserve seat:', error);
                alert('Failed to reserve the seat. Please try again.');
              }
            }
          }
          
          // Usage
          const service = new SeatReservationService('https://example.com/api');
          service.reserveSeat('A42', 'Adult');
          

          Write a PHP request handler that does $this->reserveSeat(‘A42’), and checks if it failed with a SeatNotAvailable exception

          class SeatNotAvailable extends Exception {
              public function __construct($message = "Seat not available", $code = 0, Exception $previous = null) {
                  parent::__construct($message, $code, $previous);
              }
          }
          
          class SeatReservationService {
          
              public function reserveSeat($seatNumber) {
                  // Mockup condition that the seat 'A42' is not available
                  if ($seatNumber === 'A42') {
                      throw new SeatNotAvailable("Seat $seatNumber is not available.");
                  }
          
                  // Normally here you'd interact with a database or another kind of storage
                  // to actually reserve the seat.
                  // ... (reserve seat logic) ...
                  
                  return true;
              }
          }
          
          class SeatReservationController {
          
              private $service;
          
              public function __construct() {
                  $this->service = new SeatReservationService();
              }
          
              public function handleRequest() {
                  try {
                      $this->service->reserveSeat('A42');
                      echo "Seat reserved successfully.";
                  } catch (SeatNotAvailable $e) {
                      echo $e->getMessage();
                  }
              }
          }
          

          Write a PHPUnit test for SeatReservationController, and a Jest one for SeatReservationService

          createMock(SeatReservationService::class);
                  $serviceMock->method('reserveSeat')
                              ->willReturn(true);
          
                  $controller = new SeatReservationController($serviceMock);
                  $this->expectOutputString("Seat reserved successfully.");
                  $controller->handleRequest();
              }
          
              public function testSeatNotAvailableException() {
                  $serviceMock = $this->createMock(SeatReservationService::class);
                  $serviceMock->method('reserveSeat')
                              ->will($this->throwException(new SeatNotAvailable()));
          
                  $controller = new SeatReservationController($serviceMock);
                  $this->expectOutputString("Seat not available");
                  $controller->handleRequest();
              }
          }
          
          const SeatReservationService = require('./SeatReservationService'); // Adjust the path as necessary
          
          test('reserves a seat successfully', async () => {
              const service = new SeatReservationService('https://example.com/api');
              global.fetch = jest.fn(() => 
                  Promise.resolve({
                      ok: true
                  })
              );
          
              await service.reserveSeat('A41', 'Adult');
              expect(fetch).toHaveBeenCalledWith(
                  'https://example.com/api/reserve', 
                  expect.objectContaining({
                      method: 'POST',
                      headers: {
                          'Content-Type': 'application/json'
                      },
                  })
              );
          });
          
          test('fails to reserve a seat', async () => {
              const service = new SeatReservationService('https://example.com/api');
              global.fetch = jest.fn(() => 
                  Promise.resolve({
                      ok: false,
                      status: 400
                  })
              );
          
              global.console = { error: jest.fn(), log: jest.fn() };
          
              await service.reserveSeat('A42', 'Adult');
              expect(console.error).toHaveBeenCalledWith(
                  'Failed to reserve seat:', 
                  expect.any(Error)
              );
          });
          
          • TropicalDingdong@lemmy.world
            link
            fedilink
            English
            arrow-up
            5
            arrow-down
            2
            ·
            1 year ago

            In those five or six hours, I estimate I’m able to get two or three days of work done (at my level of productivity before I started paying $20/month for ChatGPT+).

            This is very comparable to the ROI I would say that I’ve been seeing for my programming work. I feel like a super hero, or a 22 year old on adderall. I know everything I need to do for any project, but between client meetings, executive meetings, business development, meetings with the product team, mentoring, the actual amount of focused time I get is so little. I can offload a huge amount of the “I know how to do this and I’ll know if you do it wrong, but please do this for me” to the machine. This past May I took on a task that would have taken a comparable person, probably 6 months, and I knocked it out in 2.5 weeks. If you already know what you are doing, ChatGPT is steroids.

        • TropicalDingdong@lemmy.world
          link
          fedilink
          English
          arrow-up
          7
          arrow-down
          4
          ·
          1 year ago

          I would hate to see your code if you think ChatGPT’s code works great. I’m guessing you’ve never asked it do anything complex.

          Its like anything else, in that if you aren’t getting good results working with ChatGPT, you simply might not be informed enough to ask the right questions. If you don’t know what the right question is to ask, or how to form it, you’ll get poor results. I’ve been working in my field long enough to know where almost all of the bodies are buried, so I don’t fall into the kinds of traps that novices do when they really don’t understand what they are doing (although its not always obvious to them that they don’t).

          • abhibeckert@lemmy.world
            link
            fedilink
            English
            arrow-up
            3
            ·
            edit-2
            1 year ago

            If you don’t know what the right question is to ask, or how to form it, you’ll get poor results

            This. It’s taken me a while to learn how to use it effectively. I’m still learning, but I get much better results than I used to.

      • SeymoureButtocks@lemmynsfw.com
        link
        fedilink
        English
        arrow-up
        2
        arrow-down
        2
        ·
        edit-2
        1 year ago

        This. It’s incredibly useful considering it’s age and is more useful than the ego trip that StackOverflow became. Niche topics are a struggle sure, but if you know what to ask it and to check what it says, it is an amazing coding companion.