Developer Roundtable AI Edition
AI is on everyone’s mind in the tech industry, and our development team is certainly no exception. To find out how our team is really using it in the day-to-day, we invited Anna Kassinger, our Associate Director of Program Strategy, to join us at our weekly development discussion and throw some questions our way.
Anna Kassinger: Stephen, I'm going to start with you. Introduce yourself and tell me, are there any ways that you're exploring for the team to use AI tools to get better ROI?
Stephen Musgrave: Thank you, Anna. I'm Stephen, the co-founder, partner, and lead technologist at Capellic. One of the things that's always on my mind is removing unhelpful friction. With that in mind I've gone out and looked at tools like Junie, which is like an AI developer assistant agent that is built into PHPStorm. That’s a coding editor that we’ve used here for many years.
I also have looked into Rovo Dev, Atlassian’s coding agent. Rovo Dev is a command line interface [CLI] utility. Developers are already at the command line for much of their daily work, so it’s already in their flow. It's similar to Junie except it has the added benefit of having built-in communication connections to Jira and Confluence. Jira and Confluence have been at the core of how we build and maintain websites for years, so it makes sense for us to take a look. So this idea that I can, as a developer, interface with the coding agent that’s within the context of the website's code base, the Jira ticket I’m working on, and the functional specifications in Confluence – while not taking my hands off the keyboard to switch to another app – is quite attractive. Rovo Dev is also able to interface with Drupal running in our local development environments which allows it to make configuration changes using Drupal APIs via Drush. It can even run Pantheon Terminus commands so it can take care of things on Multidevs.
The core of what I'm really interested in is helping everybody here with their daily work, identifying areas where we can gain efficiencies, make the work flow, create space for creativity and bring ideas to the team.
It has everything to do with enablement. I want to be very careful when I bring these topics to the group, it can’t be a burden, that it isn’t, 'Thou shalt adopt this tool and thus become a 10x programmer.' I want to be sure that we're doing this in a way that feels comfortable, that we're trying new things, but we're not overwhelming ourselves with trying to take on too much at once. Like the work we do with our clients, this must be human-centered, it needs to flow. We’re really keeping top of mind this idea of getting the humans back to doing things humans are good at and let the machines do things the machines are good at.
Anna: Let's move to Dustin. Could you introduce yourself, your role at Capellic, and tell us the most fun tool you're using for a Capellic project that's AI related at the moment?
Dustin LeBlanc: Hi! I'm a senior Drupal developer at Capellic. For NEA, I've been using Rovo Dev to generate dev scripts for us. This is not production code, just automation tooling for manual steps that developers have to do many times. We have a bunch of documentation for processes that exist in markdown documents, which AI tools are fabulous at reading. So I've been working with it to read those documents and repeatedly do the exact same steps automatically, and then auditing the output.
Anna: It sounds satisfying to do that instead of doing a repetitive task.
Dustin: Yeah, it's one of those things where I would do this kind of work anyway, but the amount of these kinds of improvements that I can make is higher because AI makes the cost to do these kinds of things lower.
Anna: I'm interested in the idea of creativity that Stephen mentioned. Diana, I'm wondering if in any of your projects for Capellic, you've had experience with an AI tool that lets you do less repetitive work and more work that is creatively or intellectually satisfying.
Diana Casanova: I’m Diana, a senior developer with Rootstack and have been working with Capellic for almost two years. For our Drupal 11 updates, I’ve used Rovo Dev to create a spreadsheet with all of the modules installed on the website with a column indicating whether it is compatible with Drupal 11 or if it required an upgrade. The typical way to do this is with the Upgrade Status module, which is great, but the workflow around it can be time consuming.
Anna: Paul, are there any AI tools that you think are either highly overrated or underrated as assistance to developers specifically?
Paul Venuti: Oh, yes. Hi, Paul Venuti, senior Drupal developer. I would say all of them for both!
Anna: Why?
Paul: In general, my experience has been that it's best for limited tasks. So things like Diana said, that are repetitive tasks and that you already know how to do, things you just don't want to do. NEA is a multi-site, and we have something like 26 affiliates, soon to be 29 or 30 [on it]. When we make a change there to config, that change needs to be rolled out across all of the affiliate sites.
So if there are a bunch of configuration files with changes that need to be replicated across the sites, you know, we have scripts that manage taking care of that for us. But sometimes if it's just a small change to one or two files, it's easier to have an AI assistant like Junie take care of it and just say, 'Hey, I made this change. Go and replicate this change across every similarly named config file.'
So something that you can easily kind of review, where you know what the outcome is going to be. It's generally pretty great for that. Where I have found it less useful, and this is sort of like more where the hype comes in, is I have not given it like a spec for a ticket and said 'Do this.' I want to review the results carefully and I also want to make sure it's being done right. In those instances, I find it's better if I just do it myself and then lean on the AI for tasks where it's just repetitive.
Anna: So AI is like a junior assistant.
Paul: Pretty much, and I think a lot of people here would tell you the same, that if you ask AI to do something that you don't already know how to do, you aren't going to be able to trust those results, and so it becomes less efficient when you use it for that.
Stephen: I think that there is that situation sometimes, and Josh, maybe I can just tee this up for you. (Sorry Anna, I’m taking over!). It’s, 'Oh, right. I need to write some JavaScript here at this moment or I need to write a test script. I need to do something that I don't do very often and I could go research to reacquaint myself with it.’ The thinking is, 'I know when I see it if it's close', and then from there I can test it and iterate it and these are in low-risk situations, where it's going to be tested. It's not just vibe-coding straight into production. So Josh, can you maybe expound on that?
Joshua Turton: Hi, I’m Josh, I’m a senior Drupal developer here at Capellic. I have done that recently. It was on Ohio Legal Help, I believe. I am not a JavaScript developer, but I've been around long enough to know how to read the files and test the code and recognize working JavaScript. I have asked Junie to write some JavaScript to do a specific interaction, a couple of times. And I found if you explain it really well, then it will usually come back with good results. Now, some of the time it has come back with a function that does not exist. And I have to say, 'You are hallucinating that,' and try again.
What Diana and Paul were speaking of, the repetitive tasks, I’m also really on board with that. I haven't written a dependency injection [passing reusable code objects into a class instead of creating them] in PHP myself in months because there's no point. Junie, integrated into PHPStorm, can do that perfectly well, with access to one of the language models that it's using. And as a tool for tools, it’s platform agnostic. Junie can connect to GPT, it can connect to Claude, it can connect to Gemini, and several models within each of those groupings.
Generally I run it on Gemini unless I have something exceptionally complex. Then I switch over to Claude because Claude's a little more expensive but gets slightly better results in my experience. So yeah, I am stretching a little bit to do things that I know how to read and could probably write given enough time, but it's faster to ask the models to do it and see what happens. Most of the time.
Anna: Ivan, do you have a preferred large language model that you like to use or that you would be so very sad if somebody told you you had to use this one as opposed to that one?
Ivan Mendoza: Hello yeah, I am Ivan Mendoza, a Drupal developer with Rootstack working with Capellic. I use Junie and also I use ChatGPT. I prefer those. Junie has all the context because it is in the project, in PHPStorm. With other tools like ChatGPT you need to understand the issue and then send the instructions. I prefer ChatGPT because that helps me to understand the issue and the project. I prefer to give the instructions, and in some cases I use the code that it provides me, but sometimes there are functions that maybe are not available anymore in Drupal or are deprecated. So I use auto tools to understand that or in other cases I go to the Drupal page to check the APIs and whether the function that we have is correct.
Anna: I'll throw the next one out to the group. Has anybody else had an experience of using Junie or ChatGPT or any other AI assistant and felt like you learned or understood the problem better because you had to figure out how to prompt it in a different way, or just through the process of reviewing the answers or suggested path forward?
Paul: Yeah, sometimes! I may have mentioned this before to the team but it's kind of like the AI assistant almost becomes your rubber ducking partner, where just by the act of starting to like, sit down and explain the problem in a prompt, that's often enough to start you thinking about what the solution might be. So, before you even get to an answer, you're using it as that kind of a tool.
Stephen: I'm curious to see how you all react to this. There’s been a lot of talk about prompt engineering over the last couple years. Is it not just a fancy tech way of just explaining the work to be done? Isn’t it just taking the time to explain the problem well, or is that not how people think about it?
Dustin: It's exactly how I think about it. I found there's a large overlap between what I feel like I should be doing for other developers and what I need to do for this computer developer. There's maybe a difference in how I would do that because I respect a person's time and I don't respect a robot's time quite as much.
Stephen: Until I get the bill!
Dustin: The idea for why these are useful is, you're paying less for the robot's time than you would for the person's time, right, for doing the same thing? I'm always going to look at the output that it's writing and I don't care about the computer's feelings, you know. So there's no like, 'Oh, I don't want to ask them to change this because they're going to spend a bunch of hours changing it.' I don't want to get into an argument with a person about the right way to do this thing. 'No robot, I wanted it one way, you gave me some other thing, please redo it.. So I'm much more likely to just give it little tiny piecemeal things that I want it to do. I'm going to iterate on that because there's no scheduling with a robot. There's no, hey, are you free for us to do a code review on this? There's none of that.
Joshua: Dustin’s going to go down first when the robot revolution comes…
Dustin: It's literally just, 'The output was wrong. Do it this way instead. And write this to your config file so that you do this every time in the future.' It's a different experience. Whereas I'm much more considerate of taking a developer's time and nitpicking the way that they want to implement something, because it's it's a bigger investment and I don't like to make people do things that they don't absolutely have to do, you know, whereas with a robot, I have no problem telling it exactly what I want in the outcome.
Joshua: Yeah! When you're working with another developer, especially one who's less experienced, you're trying to mentor them, you're trying to guide them, help them learn and become better. But with most of the LLMs, becoming better is out of our hands, right? It's on the LLM’s dev team. We can make adjustments in their config and their agents.md file, and we can write skills for them and all of that, but by and large when we're giving feedback to a person, we’re trying to help them grow better. When we're giving feedback to one of the AI models here, it's just 'I need better results'.
Anna: Has anybody had an AI chatbot give you feedback? And what was the tone of that feedback?
Stephen: It's always very supportive, right? I mean, overly supportive, grossly supportive.
Paul: 'You are absolutely correct. I am so sorry for the confusion.'
Dustin: I've definitely done iterations like that with it. I don't always assume I'm right when I'm talking to the code agent doing this kind of stuff. I will make suggestions that are, 'Hey, did we think about this? What would happen if?' and it's pushed back and held the line on things that I asked it to do.
For example, a Rovo Dev session produced 1500 lines worth of code and I was like, 'Paul has to review this!' So I went through a couple of iterations of 'Do we need this thing? Do we need that thing? Can we cut some of this code out and use other APIs so that there's just less code in the way?' I asked it to consider a few different options and it responded 'No, it'd be better to get rid of this rather than that,' and gave reasoning for it.
Anna: So, you're saying you didn't care or worry about the robot's feelings, but the robot cared about your feelings.
Dustin: Potentially? I mean they're programmed to for sure, but it was more that I was open to multiple possible answers being correct. And sometimes just the process of asking it to consider multiple things seems to get better results. So it didn't really push back so much, I asked it to reconsider and then it did.
Stephen: I like this! I think something we heard earlier was 'you're the robot, do the thing.' That can be a bit of a trap, right? Where it's just purely transactional, pure execution. If we are exhibiting one of our human attributes of arrogance, especially if the AI is not doing the thing we want it to do, maybe it’s because it's not a good idea! Something that we talk about a lot on the team is that software always has bugs, and we tend to think about the bugs being in the code. But the bug can be all the way from the idea down to flaws in the QA process to how the code was packaged for release to how that code might like be processed in the case of JavaScript on a low-power phone with a slow connection, right? So bugs can happen anywhere, at any moment, so being skeptical of our own direction, our own biases is a great technique.
Joshua: On that note, one of the things I like about Junie is that it has the 'code mode' and the 'ask mode.’ If I'm not sure of how to approach something, I will switch it over to ask mode and prompt, 'I need it to do this. I know it can be done in Drupal. How would you recommend we do this?' And usually I get a very well-reasoned response of several paragraphs that explain that this function would do this and this hook could do that. If I like the answer, I switch it over to code mode and say, 'do that.' And then it goes and it does what it's explained to do. That gives me the opportunity to explore Drupal capabilities. Taking the AI’s knowledge and leveraging that knowledge about systems that I may not have touched before is a really great way to both expand my capabilities and my knowledge and just get stuff done.
Anna: I think it's about time to wrap up, so I'll ask one last question related to that idea of getting things done. Round robin. If AI could solve any chore or task for you either at work or in personal life, what would you wave a wand and have it be?
Dustin: I'm going to go with a really boring answer and just say AI, take my garbage out so I don't have to.
Ivan: Okay, well, I think that is to iron the clothes.
Anna: You know what? That sounds like a task that is quite repetitive and you could oversee effectively. Oh, I like it.
Stephen: I just want you all to know I've never seen Ivan show up to work in a pressed shirt!
Anna: Well, that's because the AI isn't here yet! All right, Paul. Work or life tasks that AI solves for you?
Paul: Yeah. Something to do with grocery shopping and picking out good produce that I would actually want. I'm picky about broccoli, right? My wife's picky about lettuce. It'd be great if AI could do that and then just do the shopping and then I suppose unpack it and put it all away.
Dustin: Then you can blame the AI when the lettuce is soggy!
Diana: I think two things. The first one is mechanical: washing the dishes. I hate that. The other one is to organize my days and my ideas—not just the work, the new ideas that I have for a new business or something like that. But I have this [holds up paper journal] and I have every day added and added to it.
Anna: That seems like a nearer term possibility. I like it. It doesn't involve any building or mechanical devices. Stephen, what wish fulfillment could AI bring for you?
Stephen: It's really what Diana talked about. There's just so much information coming at us. There's information coming out of our heads. We're awash in ideas. How do we move things forward in a satisfying way? We have these information silos, whether it be Basecamp or Jira or Confluence or Slack or our dev discussions, where the transcript ends up in Google Docs, and there's Gmail… it's everywhere. An AI assistant for wayfinding would be great.
So, just to circle around again, I’m talking about unhelpful friction again. I don't think it’s useful to spend 10 minutes trying to collect knowledge. To be able to have a 'Capellic bot' that I can just ask for those details is very interesting to me.
Joshua: Most of us come back around to the starting point of our conversation. What are you doing with it right now? We want it to 'make things easier.' Take away the monotony and the drudgery and the hard repetitive tasks that we know how to do and don't want to anymore.
I would love to see AI like Paul said, do the grocery shopping or more importantly plan the menus for the week and then assemble the grocery list. That would be something that I haven't tried, and maybe I should. 'I am a capable cook. We like certain things and have some food allergies. Plan me four days of meals for our household. Some of them are teenagers, so multiply their portions by 1.7.' That would be great.
Anna: Well, thank you all for letting me crash your meeting. It's great to be taught by you all and I'm sure the conversation will continue!
Joshua: Thank you, Anna!
Editor's note: This interview was transcribed by Gemini, an AI tool. It has been edited for clarity by humans and trimmed down for length.
