Not long ago, Orion Star released the Orion-14B-Base, which attracted my attention. Its core attraction is: the long context version supports ultra-long text, which has excellent results in the length of 200k tokens, and can support up to 320k at the longest, which is quite durable in the current open source models, which is 10 times the number of tokens that chatglm3-6b-32k bears, which is comparable to yi-34b-200k, but the trained chat version is convenient for interactive data analysis. Of course, the graphics card is also the main consideration, and the 34B graphics card resource requirements are really high, and the official quantitative version has not been released. In mainstream public benchmark evaluations, the Orion-14B series models performed well, significantly outperforming other models with the same parameters in a number of indicators.
To get back to the point, LangChain is the current mainstream LLLM programming framework, if you want to develop an LLM-based application, it has all the components you need, just use it directly; Even for the general application process, it already has a built-in standardization scheme using the concept of chain (the origin of chain in langchain).
But a bug was found in the integration of langchain and orion-14b-longchat, and it has been running unsuccessfully:
I get the error you can **oid this prompt in future by passing the argument trust remote code=true.
In Langchain StartupNo matter how you set it, py doesn't take effect, and after careful analysis of the error file, it turns out that it's a model adapterThere is a problem with py, the reason is that the langchain-chatchat project is not compatible with it, and because longchat appears in modelname and path, langchain extracts the longchatadapter adapter according to the lmsys longchat-7b-16k model when the model is adapted, and it does not pass trust remote code=true.
Find the problem and there you have a solution:
at FastChat Model Model Adapterpy.
First, directly modify the longchatadapter adapter and pass trust remote code=true, but this will affect the use of the longchat-7b-16k model;
The second way: write a new OrionChatAdapter, according to the parameter requirements of Orion, pass trust remote code=true, and register the adapter at the bottom, then pay attention to put it in front of LongchatAdapter, because it is matched in order.
After modifying, saving, and you can use it smoothly.