call openAI provide github PR feedback hackathon

For Opentables 2023 October Hackathon I proposed integrating with openAI api to get feedback on github PRs . I'd already used the chatGPT interface to get feedback on code or write tests and it's good.

I know there are github Actions out there which do similar but I wanted to hack a solution for our companys setup and also to learn. Another engineer thought it was a fun idea and joined me in this hack. 

My initial idea was to: 

  1. create a github Action which triggers off PRs and 
  2. call github to get the PR code
  3. call the openAI chat api and passes it the code
  4. call github to add a comment to the PR
One challenge with this was that would be running externally and would not have access private data needed to access things.
So we pivoted to running the steps inside our CI build system instead. This gave us github triggers such as PR opened and also secure access. To get access to the PR number we had to enable a feature in the CI, but I did and it worked!

We used bash scripts and I was able to use curl to call to github to get a PR diff and also use curl to add a comment to a PR. Nice!
But calling openAI over bash and escaping data was a real problem and not working. Half way through day 2 of the hack and we did't have a working solution! Not good.
So pivot number 2: onwards to python

We rewrote the hack in python and within a few hours everything was basically working. 
When open a PR the code diff is retrieved, send to openAI, get back feedback, add as PR comment and it worked! fantastic

Our solution is very basic and assumes a happy case of a PR adding a new file. 
  • How would you support multiple pushes to a PR? Don't want to keep calling openAI and duplicating AI comments.
  • A PR of a totally new file returns a github diff of the new file, but what about a PR with some changes to existing files(s)? What would you get and pass to openAI to make sense of it?
Shortly after our Hackathon at the Github universe conference they announced github PR copilot support. They stole our idea! lol
Not surprised that is added. Have yet to use it myself.

The day is close when the first PR reviewer will be copilot and engs won't even look at a PR until the AI feedback is addressed (almost like a linter). How much further is the day when the AI writes the code as well...and no need to review its own code (scary). But that's a different discussion.


Appendix
our call to openAI was basic using model: gpt-3.5-turbo, role: 'user' and content: <the diff text> and chat completions api  and response is in json content prop

Comments

Post a Comment

Popular posts from this blog

deep dive into Material UI TextField built by mui

Image
The Material Design system is a design system used to create consistent digital experiences across platforms such as android, web and ios. Includes: a color system a typography system a shape system a grid system it addresses motion and lots more Also includes Material Theming which makes it easy to customize Material Design to match your brand. Material Design is not just a set of guidelines it is a whole ecosystem. Material itself stopped maintaining it's react implementation and recommends opensource versions.  A very popular open source implementation is  MUI ,  a react component library  built to implement Material Design . Bills itself as the "worlds most popular React UI framework" and at this time has 1.2M weekly downloads on npm and used by over 750k projects on github (did I mention it's very popular?).  Note: there are other react component libraries implementing bootstrap and ant design for example, so up to you and your design team on what you choose. 
Read more

angular js protractor e2e cheatsheet

Many from the protractor test suite category detailed task syntax example load page load a page browser.get('yoururl') browser btn actions browser back/forward browser.navigate().back() OR browser.navigate().forward() access elements on page See Protractor API list and examples from Protractors test suite access by angular ngmodel name (must match ngmodel in the html) this is the preferred approach element(by.model('person.name')); access by the css id on the page e.g, #user_name (but this can be brittle) element(by.id('user_name')) access angular binding to get generated value i.e. for {{}} element(by.binding('person.concatName')); access angular repeat list data (must match repeat stmt in the html) See Protractor test suit l
Read more

react-router v6.4+ loaders, actions, forms and more

react-router  react-router is used for in app single page app navigation, we configure routes for pages in the app to map to components react-router has over 1bn downloads, its an industry standard (but there are others e.g. Nextjs router) many apps use react-router, many on v5.x the new react-router 6.4.x the newest major version of react router v6.4 is a significant enhancement which adds a lot of new features. react-router v6.4 offers new features to load data (loaders) and handle actions (actions).  It's done gone all Remix-y which makes sense because the same guys wrote both. eliminates a lot of boilerplate code to fetch data in useEffect() and set various loading states put your data load and update logic in react-route components instead of custom code unlike Remix, react-router v6.4 is still just all on the client (not server like Remix) paradigm shifts Lets start with a couple of high level ideas; "back to the future" so to speak OR "what's old is new ag
Read more