So, in my search for a less restrictive AI LLM writing tool, I came across references to FreedomGPT. It’s an offline large language model(LLM) tool that promises completely offline, private, unrestricted, unencumbered, no holds barred LLM to compete with the likes of ChatGPT.
So I decided to give it a look, a download, and a review.
What FreedomGPT Promises
So the biggest selling point is the entirely cloud-free LLM that isn’t tied down by corporate restrictions on what it will and won’t talk about. Depending on your field of study, research, etc. this can get pretty frustrating. Just recently I had ChatGPT refuse to do something and threaten to report me for asking it to write an image description about a hybrid animal(apparently it thought I was conducting inhumane experiments on animals).
In theory, this would also help to lift some of the restrictions common to cloud LLMs like message length. Despite the best prompt engineering, it’s near impossible to get most LLMs to write more than around 700 words before giving up. But, on your own hardware, there’s no limiting factor to tell it to stop and save the electricity.
It also promises the ability to use varying LLM models including llama and alpaca, but neither seems to show any real promise currently. Here’s why.
My Honest Review of FreedomGPT
While I like the idea of a totally offline LLM tool without so many restrictions, this one so far fails to deliver any useful results. I think it’s just too underpowered to have any meaningful use in even the most basic content generation.
Don’t get me wrong, it does write in understandable English. But it’s style is beyond mechanical, it’s knowledge is severely limited, and it’s just plain DUMB when it comes to following instructions.
You ever spend time talking with an Alzheimers sufferer, who every 20 seconds forgets what they are saying and starts telling the same old story? That’s pretty much the feeling you’ll get. It’s nowhere near such a refined tool as even Bard or any of the other less intelligent cloud-based LLM chat bots.
An Example of FreedomGPT’s Limits
So, as I test, I decided to ask it to write me something about data recovery. It’s a field I used to work in, so I can easily fact check it just using my brain and years of experience in the field.
Here’s the prompt I decided to start it off with:
Write me a full book on the topic “Data Recovery For Dummies”. Be as thorough and detailed as possible. I would like it to be at least 100 pages in length.
Simple enough right? I gave it no technical details and the freedom to write about whatever knowledge of the subject it has. Also, I gave clear instructions that I was looking to write a lenghty book, not just a quick blog post or a few paragraphs.
Because it runs entirely on your local computer, there are no specific handicaps set on how long a response it should be able to generate. So in theory, if I let it eat enough CPU time and electricity, it’s capable of writing a full book.
The “Book” It Wrote
Here’s what it spits out on the first run:
Chapter 1: Introduction to Data Recovery
Data recovery is the process of salvaging valuable data from damaged, failed or corrupted storage devices. It is important to note that data recovery is not the same as backup, as it focuses on recovering data from damaged storage devices rather than preventing data loss in the first place. In this chapter, we will discuss the different types of data recovery, the importance of regular backups, and the tools and techniques used to perform data recovery.
Chapter 2: Understanding Storage Devices
In this chapter, we will discuss the different types of storage devices, their advantages and disadvantages, and the different file systems used to store data on them. We will also discuss the different types of storage media, such as hard drives, solid state drives, and optical storage media.
Chapter 3: Understanding Data Corruption
Data corruption is a common problem that can occur due to hardware or software failure, malicious attacks, or even user error. In this chapter, we will discuss the different types of data corruption, the causes, and the techniques used to recover corrupted data.
Chapter 4: Understanding Backups
Backups are an essential part of data recovery, as they provide a reliable way to restore data in the event of a disaster or hardware failure. In this chapter, we will discuss the different types of backups, the advantages and disadvantages of each, and the tools used to perform backups.
Chapter 5: Tools and Techniques for Data Recovery
In this chapter, we will discuss the different tools and techniques used to perform data recovery, such as file system analysis, data carving, and forensic analysis. We will also discuss the different steps involved in the data recovery process, such as data collection, data analysis, and data restoration.
Chapter 6: Recovering Data from Common Storage Devices
In this chapter, we will discuss the different techniques used to recover data from common storage devices, such as hard drives, solid state drives, and optical storage media. We will also discuss the different tools used to perform data recovery from these devices, such as data carving and forensic analysis.
Chapter 7: Recovering Data from Servers and Network Storage
In this chapter, we will discuss the different techniques used to recover data from servers and network storage, such as FTP, NFS, and SMB. We will also discuss the different tools used to perform data recovery from these devices, such as data carving and forensic analysis.
Chapter 8: Recovering Data from Cloud Storage
In this chapter, we will discuss the different techniques used to recover data from cloud storage, such as Amazon S3, Google Cloud Storage, and Microsoft Azure. We will also discuss the different tools used to perform data recovery from these devices, such as data carving and forensic analysis.
Chapter 9: Conclusion
Data recovery is an important part of digital forensics, and is essential for preserving valuable data in the event of a disaster or hardware failure. In this chapter, we will discuss the importance of regular backups, the tools and techniques used to perform data recovery, and the importance of data security.
Now it’s not technically wrong in anything it says. Honestly, it just didn’t really SAY anything at all. It just came up with a most basic table of contents for some chapters.
Even in how it described each chapter, it’s already starting to sound redundant. It seems to love three-item, comma-separated lists (e.g. “such as data collection, data analysis, and data restoration”) which it seemed to recycle with little to no significant variations.
But, I assume, no problem. Maybe it’s just a table of contents and hasn’t actually provided any details in the list of chapters. So I go ahead an give the follow-up prompt:
“That’s a good table of contents. Now begin by writing chapter 1 in its entirety.”
It’s reply:
Chapter 1: Introduction
In this chapter, we will introduce the concept of artificial intelligence and its applications in the real world. We will explore the history of AI, its current uses, and potential future applications. We will also discuss the potential benefits and drawbacks of AI and its implications on our lives.
Failure!
As you can see, in just two prompts the LLM already forgot entirely what we were talking about. It began writing a chapter about AI instead of about data recovery.
Why doesn’t FreedomGPT measure up?
There’s a few reasons I can think of.
- Training size. The models available, llama and apaca, are only around 4Gb each. That’s not tiny for a language model, but I’d imagine it’s only a small fraction of the big brain used by Open AI’s ChatGPT.
- Hardware acceleration. I noticed that FreedomGPT is basically only using CPU acceleration. My two GPUs were nearly idle during text generation. I happen to know that other cloud based LLMs are using enterprise NVIDIA cards with thousands of GPU cuda cores. It’s hard to imagine that anything using only CPU cores could compete with the power of all those cores algorithmically calculating the probabilities.
I sincerely hope that in the future this project becomes a viable alternative. Even if it’s just a nice offline copyright tool with 1/10th the knowledge of the big cloud providers, it’d be nice to have an offline tool that can generate useful language text.
But, for now, we all need to keep paying $20/mo for ChatGPT plus and hope they don’t make the same mistakes as Bing and leak our chat logs out onto the internet search.
Till next time, let’s keep watching the AI overlords.