The field - Intent Recognition
Turns out that’s what I should look into. This field of NLP deals with figuring out what a user wants from their prompt. It also tags tokens in the input string, which can then be used for further computations or whatever.
The first option - overkill?
There’s a model called BERT. It’s a fancy model whose piece de resistance is the fact that it can factor in context to figure out what’s going on. What’s more, is that it coems pre-trained? And you only need one layer of problem specific training to make it tailored to your needs. Which is pretty cool. I found this article, which shows the massive differences that BERT makes in intent recognition, by Walmart Labs.
This does mean that I’ll need to use, for example, TensorFlow, perform one layer of training, and then compress it into a TFlite model. And then use the Dart runtime for it. Mmm…not very appealing.
The second option
I’m not too sure about this…but what if I just directly tag entities?
And then maybe depending on the entities apply, appropriate regex? I mean, I’m not trying to build chat here, am I? It certainly seems far less involved, and I don’t need to use anything exterior for that.
(Edited)
A third option?
Instead of all that…what if I make a text classifer specific to the capabilities I want to bake into the software, and one extra class to say we don’t do that just yet - in case the prompt is something we don’t do? And then have different regex for each class?
I’m still not sure how I’ll make my own classifier in Dart, but that sounds a whole lot more doable than the BERT approach.
Let’s see. I’m torn between the last 2 options, since BERT seems like total overkill. I also wonder what the speed difference between the 3 approaches is. Maybe BERT takes more time? Dunno.