Never did this before, so I was asking Q in the AWS docs how to do it.
It refused to help, as it didn't answer security related questions.
thank.
one of my questions about a login form also tripped a harassment flag
I suspect instead the people training these models have identified areas of questioning where their model is 99% right, but because the 1% wrong is incredibly costly they dodge the entire question.
Would you want your LLM to give out any legal advice, or medical advice, or can-I-eat-this-mushroom advice, if you knew due to imperfections in your training process, it sometimes recommended people put glue in their pizza sauce?
Amazon claims the Titan model is suitable for: "Supported use cases: RAG, agents, chat, chain of thought, open-ended text generation, brainstorming, summarization, code generation, table creation, data formatting, paraphrasing, rewriting, extraction, and Q&A." (it is not, lol)
So sure, the LLM occasionally pranks someone, in a way similar to how random Internet posts do; it is confidently wrong, in a way similar to how most text on the Internet is confidently wrong because content marketers don't give a damn about correctness, that's not what the text is there for. As much as this state of things pains me, general population has mostly adapted.
Meanwhile, people who would appreciate a model that's 99% right on things where the 1% is costly, rightfully continue to ignore Gemini and other models by companies too afraid to play in the field for real.
A random person on the Internet often has surrounding context to help discern trustworthiness. A researcher can also query multiple sources to determine how much there is concensus about.
You can't do that with LLMs.
I cannot stress strongly enough that direct comparisons between LLMs and experts on the Internet are inappropriate.
In this context, I very much agree. But I'd like to stress that "experts on the Internet" is not what 99% of the users read 99% of the time, because that's not what search engines surface by default. When you make e.g. food or law or health-related queries, what you get back isn't written by experts - it's written by content marketers. Never confuse the two.
> A researcher can also query multiple sources to determine how much there is concensus about.
> You can't do that with LLMs.
A person like that will know LLMs hallucinate, and query multiple sources and/or their own knowledge, and/or even re-query the LLM several times. Such people are not in danger - but very much annoyed when perfectly reasonable queries get rejected on the grounds of "safety".
What is the consensus on liability in case of regular web search? Your comment made me realize that I never thought much about it in 20+ years of using the Internet; I kind of always assumed it's all on the user.
Have you never noticed those "google has removed some results to comply with the DMCA" notices?
Also fun fact, Titan's image generator will refuse any prompt that references Bezos because it "violates content policy"
If you want to do something useful on bedrock use Claude
Claude Opus is supposedly only available in us-west-2, but is listed as "Unavailable" for me (Sonnet and Haiku are available). Cohere's Command R+ is also available and while less capable, for instruction following, I believe its superior to Anthropic's models. There's also Llama 3 70B Instruct and Mistral Large, both which are good for general tasks.
For those that haven't been closely following/testing the models available, I think Artificial Analysis' Quality vs Price charts isn't too bad a place to start https://artificialanalysis.ai/models although if you have specific tasks, it's best to eval some models are surprisingly good/bad at specific things.
Titan appears to be bad at everything though.
My experience recently is that its actually noticeably better for instruction following than Claude, but can be finicky if you're not careful about adhering to the prompt template. But between the RAG and multi-step tool use capabilities, even if it was slightly worse on the instruction-following side of things I'd still say, as you do, thats its much better than Claude on average.
Agree on titan as well. I recently was forced into a meeting with our AWS TAM, and they kept shoehorning Q into every conversation. I held my tongue knowing that titan was the model powering it under the hood.