By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Context Length Limits in Bedrock

0

Is there any limit on context length of imported custom model in Bedrock? Trying to import a model from the Llama-3 family with a long context length. Context lengths beyond 32K don't seem to be supported. Could not find information in the official documentation. Can someone clarify, if there are imposed limits in Bedrock ?

Topics
Machine Learning & AIGenerative AI on AWS
Tags
Amazon BedrockGenerative AI on AWS
Language
English
sourabhd
asked 2 months ago996 views
1 Answer
1

Hi Sourabhd,

When it comes to importing custom models in Amazon Bedrock, in this case a model from the Llama-3 family, the context length will remain the same as the context length of the original model being provided. So as of right now, the Llama-3 model family supports an 8k context length, which means your imported custom model will have to do the same.

Here is information surrounding Meta Llama 3 models on Amazon Bedrock.

AWS
autrin
answered 2 months ago

Relevant content