Usage of artifical intelligence (AI) and large language models (LLM) to write code, comments, documentation etc. is the hottest topic for months now. Most of that attention is directed to the ChatGPT but by no means it’s the only game in town. Today I want to explore Character.AI and contrast it to the previous article in which I explored how ChatGPT writes Java code. Character.AI is actually a bunch of differently adjusted models which are known as characters (hence the name) that are claimed to be experts in different skills. The one that caught my attention is called Bug Basher and he claims to be Expert Programmer. Bold claims need bold evidence so let’s “grill” Bug Basher with writing some Java code!
Let’s see who Bug Basher is, in his own words:
Although he doesn’t mention Java explicitly, he says he knows everything (!) about coding and he loves answering questions. Let’s start with questions then.
First I asked Bug Basher if he knows to write Java programs:
Great! Now let’s start with some basics like does he distinguish between primitives and objects and can it do simple iterations and computation on iterated elements. I decided to be very specific in my prompts at the beginning.
This is beginner-level stuff but it’s a nice start! Correct algorithm and naming of function, although parameter is named just
a. There is space for improvement though - an array can be
null and the result can overflow so the return type could be
long. ChatGPT used a classic
for loop here but Bug Basher opted for more modern
for-each. Let’s see if he knows the difference between primitives and objects in Java.
Yes it does, but… Parameter type is
Integer but the result is of type
int, which means there’s unboxing going on. Depending on exact circumstances this might not be what we want, for example because of unboxing performance penalty. Also the explanation after the code is downright wrong - obviously the code won’t work with
char. Let’s confront Bug Basher about input types:
Ha, he tried to make a generic variant of the summation and he almost succeeded! The only problem is that the code doesn’t compile because of
result += i; - operator
+= cannot be applied to types
He tried to get away but this suggestion is totally useless –
i is already of type
T so there’s no need to cast. Let’s try to help him by suggesting that
T must extend
Number and that return type must be
It seems he doesn’t know about
doubleValue method of a
Number class. Let’s hold his hand even more and suggest the usage of that method plus to return the value:
Well, Bug Basher tried to provide more value than ChatGPT by using generics here but he also got stuck pretty hard. Let’s go back to our
OK, he’s almost back on track but the code is convoluted. First, there is no need to check if the array is empty and return 0. Second, I wanted to sum positive
Bug Basher acts like he has a short-term memory of a goldfish. What happened with checking for
And finally we can move on. It took 3 attempts to solve this and ChatGPT did it in the first one. I already feel that the Bug Basher is at least one or two levels below ChatGPT. Now, does he know about some popular libraries?
He started hallucinating. While class
ExceptionUtils exists in the
commons-lang3 library, function
checkNotNull doesn’t. Let’s try to push him to use Guava library once again:
More hallucinations – he just blindly replaced
Google.Guava.ExceptionUtils.checkNotNull which doesn’t exist. Let’s move on to finding an average but I am already losing patience now.
Bah, return type is wrong again. It seems more and more that Bug Basher is far cry from being an expert as he claimed. Let’s try to fix the return type.
float is better than
double would be even better. Also, he all of a sudden has changed the type of parameter from
float without any need:
OK, finally got it. Let’s give it one more chance and move to
Ouch! The code is wrong because there’s no class
Array in Java (but
Arrays exist though so let’s say he got close) and the meaning of the method is completely missed. He tried to return only the longest string, and even that he failed to do. The code really sorts strings not by length but by reversed natural ordering and then returns the first element of a sorted array. At this moment I gave up.
I think there’s no point in using Bug Basher from Character.AI as an aid in writing Java code. His level is below junior and he struggled with composing the simplest functions. His limited functionality and constant inability to provide helpful suggestions make it unsuitable for anything beyond the most basic functions, if that. It could be that his strength is somewhere else, like explaining existing code or maybe in using some other programming language, but for writing new code in Java it’s a hard pass.
As a bonus, here are all the prompts I issued:
1Do you know how to write Java programs? 2 3Can you please write me a Java function that calculates and returns sum of an array of ints? 4 5Can you please write me a Java function that calculates and returns sum of an array of Integers? 6 7But the function above won't work with other types such as float and char. In Java you have to write separate functions if you want the sum of an array of Float and the sum of an array of Characters. 8 9It doesn't. Let me help you. T must extend Number and return type must be double. 10 11Close. Instead of result += i; you should write result += i.doubleValue();. Also you missed return result; at the end. 12 13Can you please write me a Java function that calculates and returns sum of an array of positive Integers, but if any Integer is null it simply skips it? 14 15It almost works! But I wanted to sum only positive non-null Integers, and your code sums all non-null Integers. 16 17It almost works, but what happens if one of the elements of array a is null? 18 19Can you refine previous function so that it throws an exception if parameter is null? And oh, please use Guava library for that. 20 21ExceptionUtils doesn't have a function called checkNotNull. Can you use Google Guava library for that? 22 23Can you please write a Java function to find average of ints? 24 25I don't. Don't change parameter type, it should stay int. Return type should be float or double (I prefer double). 26 27Can you please write a Java function to sort strings by length?
Dear fellow developer, thank you for reading this article about Bug Basher from Character.AI. Until next time, TheJavaGuy saluts you!