Nous Research

Last week, we released our first step in validating and independently confirming the claims of the Bitnet paper, a 1B model trained on the first 60B tokens of the Dolma dataset. Comparisons made on the Weights & Biases charts below are between the Bitnet implementation and a full FP16 run (all hyperparameters equivalent). Model: https://lnkd.in/evCdAgKH Weights and Biases: https://lnkd.in/etWtQ6n3

5 months ago, Andreessen Horowitz（A16Z）announced its support for the Open Source AI Initiative, which includes Nous Research and a friend of mine. Half a month ago, Nous Research just announced the closing of $5.2 million seed financing round. Today, Nous Research are announcing the latest project, an effort to provide a new evaluation system for open source models. Traditional benchmarking leans heavily on public datasets which can be easy to game and often lead to superficial score improvements that mask true model capabilities (or lack thereof), misdirecting efforts towards meaningless score chasing. To achieve this, Nous Research built a subnet on Bittensor, a decentralized network for AI projects. This system allows people to submit your finetuned model, and then validators will evaluate all models submitted against freshly generated data from the Cortex subnet. The Cortex subnet is a dynamic source of fresh synthetic data, continuously generated by GPT-4, which allows for fresh and unpredictable high quality data to test models against, and ensures a fair and accurate consensus on which model most closely mirrors GPT-4's performance. This new system aims to celebrate and reward creators of public, open source models that genuinely meet user needs, through the incentive structures built into Bittensor. Learn more on the following pages: Nous Subnet Leaderboard： https://lnkd.in/gHjM6Equ Nous Subnet Repository：https://lnkd.in/gSzcWVTH Bittensor：https://bittensor.com Cortex Subnet Repository :https://lnkd.in/guBmfuNu #ai #artificialintelligence #opensource #gpt #decentralized

🎉 NousResearch just released Nous-Hermes-2-Mixtral-8x7B 🏆 Potentially the best open-source LLM: high-quality merges incoming! 🥇 First fine-tuned version of Mixtral 8x7B that outperforms the official Mixtral Instruct 📅 Trained on >1 million samples generated by GPT-4 + other open-source datasets They released an SFT and a DPO versions of this model + GGUF versions. Impressive work, true MoEs like Mixtral are tricky to fine-tune. This is going to be a very popular model. 🤗 SFT: https://lnkd.in/g5ZT7B7b 🤗 DPO: https://lnkd.in/g8hPubpz

Nous Research is excited to announce the closing of our $5.2 million seed financing round. We're proud to work with passionate, high-integrity partners that made this round possible, including co-leads Distributed Global and OSS Capital, with participation from Vipul Ved Reddy, founder and CEO at Together AI, Yonatan Ben Shimon, founder at Matchbox DAO, and several angel investors including Balaji Srinivasan, entrepreneur and investor, Thibaud Zamora, entrepreneur and investor, Alex Attallah, founder at OpenRouter and OpenSea, Chris Prucha, investor and founder at Notion, Sahil Chaudhary, founder and CEO at Glaive AI, and Gavin Uberti, founder and CEO at Etched AI. Nous has historically been a volunteer project- with this investment, we can empower a small group of our most dedicated members to join us in bringing a composer for AI orchestration to all, Nous-Forge, in 2024! The importance of pursuing open source AI today cannot be overstated; Nous will continue open-source research in LLM Architecture, Data Synthesis, Simulation, & Agent Engineering, amongst other areas. We welcome your ideas and experiments here, always (no matter how out-of-the-box they may be). Let's talk: https://lnkd.in/eQ3ZvJm4

We’re extremely proud to contribute to post-transformers architecture with the collaborative release of the StripedHyena family of models alongside Together.

Our latest release in the Capybara series is now available on HuggingFace. Trained on Yi-34B for 3 epochs, this new model is our first 34B parameter fine-tune, utilizing Yi's 200k context length. https://lnkd.in/ePh5Btmk

AI Agent memories represented by a 'neuron pathway'; the tree expands with memories as the Agent is used. From @otisllm at Nous!

AI Agent memories represented by a 'neuron pathway'; the tree expands with memories as the Agent is used. From @otisllm at Nous!

𝐈𝐧𝐭𝐫𝐨𝐝𝐮𝐜𝐢𝐧𝐠 𝐎𝐩𝐞𝐧 𝐇𝐞𝐫𝐦𝐞𝐬 𝟐, a continuation of the Hermes series of models, now built on Mistral 7B! The Hermes 2 model was trained on 900,000 instructions, and surpasses all previous versions of Hermes 13B and below, and matches 70B on some benchmarks! Hermes 2 changes the game with strong multiturn chat skills, system prompt capabilities, and uses ChatML format. It's quality, diversity and scale is unmatched in the current OS LM landscape. Not only does it do well in benchmarks, but also in unmeasured capabilities, like Roleplaying, Tasks, and more. Check out @Teknium for more context on his creation process, the vision, & how Hermes came to life 😊 https://lnkd.in/ekYRsEEd