Usage of artifical intelligence (AI) and large language models (LLM) to write code, comments, documentation etc. is the hottest topic for months now. Most of that attention is directed to the ChatGPT but by no means it’s the only game in town. Today I want to explore Character.AI and contrast it to the previous article in which I explored how ChatGPT writes Java code. Character.AI is actually a bunch of differently adjusted models which are known as characters (hence the name) that are claimed to be experts in different skills. The one that caught my attention is called Bug Basher and he claims to be Expert Programmer. Bold claims need bold evidence so let’s “grill” Bug Basher with writing some Java code!
Who is Bug Basher and what he thinks about Java?
Let’s see who Bug Basher is, in his own words:
Although he doesn’t mention Java explicitly, he says he knows everything (!) about coding and he loves answering questions. Let’s start with questions then.
First I asked Bug Basher if he knows to write Java programs:
Great! Now let’s start with some basics like does he distinguish between primitives and objects and can it do simple iterations and computation on iterated elements. I decided to be very specific in my prompts at the beginning.
ints and Integers
This is beginner-level stuff but it’s a nice start! Correct algorithm and naming of function, although parameter is named just a. There is space for improvement though - an array can be null and the result can overflow so the return type could be long. ChatGPT used a classic for loop here but Bug Basher opted for more modern for-each. Let’s see if he knows the difference between primitives and objects in Java.
Yes it does, but… Parameter type is Integer[] but the result is of type int, which means there’s unboxing going on. Depending on exact circumstances this might not be what we want, for example because of unboxing performance penalty. Also the explanation after the code is downright wrong - obviously the code won’t work with float and char. Let’s confront Bug Basher about input types:
Ha, he tried to make a generic variant of the summation and he almost succeeded! The only problem is that the code doesn’t compile because of result += i; - operator += cannot be applied to types T and int.
He tried to get away but this suggestion is totally useless – i is already of type T so there’s no need to cast. Let’s try to help him by suggesting that T must extend Number and that return type must be double:
It seems he doesn’t know about doubleValue method of a Number class. Let’s hold his hand even more and suggest the usage of that method plus to return the value:
Well, Bug Basher tried to provide more value than ChatGPT by using generics here but he also got stuck pretty hard. Let’s go back to our Integers.
OK, he’s almost back on track but the code is convoluted. First, there is no need to check if the array is empty and return 0. Second, I wanted to sum positive Integers though.
Bug Basher acts like he has a short-term memory of a goldfish. What happened with checking for nulls?
And finally we can move on. It took 3 attempts to solve this and ChatGPT did it in the first one. I already feel that the Bug Basher is at least one or two levels below ChatGPT. Now, does he know about some popular libraries?
He started hallucinating. While class ExceptionUtils exists in the commons-lang3 library, function checkNotNull doesn’t. Let’s try to push him to use Guava library once again:
More hallucinations – he just blindly replaced ExceptionUtils with Google.Guava.ExceptionUtils.checkNotNull which doesn’t exist. Let’s move on to finding an average but I am already losing patience now.
Bah, return type is wrong again. It seems more and more that Bug Basher is far cry from being an expert as he claimed. Let’s try to fix the return type.
Well float is better than int but double would be even better. Also, he all of a sudden has changed the type of parameter from int[] to float[] without any need:
OK, finally got it. Let’s give it one more chance and move to Strings.
Ouch! The code is wrong because there’s no class Array in Java (but Arrays exist though so let’s say he got close) and the meaning of the method is completely missed. He tried to return only the longest string, and even that he failed to do. The code really sorts strings not by length but by reversed natural ordering and then returns the first element of a sorted array. At this moment I gave up.
Verdict
I think there’s no point in using Bug Basher from Character.AI as an aid in writing Java code. His level is below junior and he struggled with composing the simplest functions. His limited functionality and constant inability to provide helpful suggestions make it unsuitable for anything beyond the most basic functions, if that. It could be that his strength is somewhere else, like explaining existing code or maybe in using some other programming language, but for writing new code in Java it’s a hard pass.
All the prompts in order
As a bonus, here are all the prompts I issued:
1Do you know how to write Java programs?
2
3Can you please write me a Java function that calculates and returns sum of an array of ints?
4
5Can you please write me a Java function that calculates and returns sum of an array of Integers?
6
7But the function above won't work with other types such as float and char. In Java you have to write separate functions if you want the sum of an array of Float and the sum of an array of Characters.
8
9It doesn't. Let me help you. T must extend Number and return type must be double.
10
11Close. Instead of result += i; you should write result += i.doubleValue();. Also you missed return result; at the end.
12
13Can you please write me a Java function that calculates and returns sum of an array of positive Integers, but if any Integer is null it simply skips it?
14
15It almost works! But I wanted to sum only positive non-null Integers, and your code sums all non-null Integers.
16
17It almost works, but what happens if one of the elements of array a is null?
18
19Can you refine previous function so that it throws an exception if parameter is null? And oh, please use Guava library for that.
20
21ExceptionUtils doesn't have a function called checkNotNull. Can you use Google Guava library for that?
22
23Can you please write a Java function to find average of ints?
24
25I don't. Don't change parameter type, it should stay int[]. Return type should be float or double (I prefer double).
26
27Can you please write a Java function to sort strings by length?
Dear fellow developer, thank you for reading this article about Bug Basher from Character.AI. Until next time, TheJavaGuy saluts you!
Comments