Signal feed
AI stories, scored and filtered.
Live items from our monitored sources, filtered for signal and annotated with a recommended posture for enterprise leaders.
997 stories
- 11 JulResearch
2023-7-9 arXiv roundup: LLMs ignore the middle of their context, MoE + instruction tuning rocks
Davis Summarizes Papers
Research indicates LLMs struggle with information in the middle of long contexts and that Mixture-of-Experts (MoE) models improve with instruction tuning.
Why it matters
The 'lost in the middle' phenomenon for long context windows directly impacts retrieval-augmented generation (RAG) effectiveness, while MoE advancements offer new pathways for highly efficient specialized models.
Hype4/10 - 2 JulResearch
Models generating training data: huge win or fake win?
Davis Summarizes Papers
Research investigates if LLMs synthesizing training data for fine-tuning other models improves performance or introduces bias, showing mixed results.
Why it matters
Synthetically generated training data, while promising for data scarcity, introduces novel risks around model drift and hallucination that demand robust validation frameworks.
Hype6/10 - 20 JunResearch
Have we hit a statistical wall in LLM scaling? - 2023-6-18 arXiv roundup
Davis Summarizes Papers
Recent research questions the indefinite scaling laws of LLMs, suggesting statistical limits may be approaching for performance gains.
Why it matters
The potential deceleration of LLM scaling means your build-vs-buy strategy for frontier models may shift towards proprietary fine-tuning and smaller, more efficient models for specific tasks.
Hype4/10 - 14 JunResearch
2023-6-11 arXiv: Training on GPT outputs works worse than you think, but training on explanations works great
Davis Summarizes Papers
Research indicates training smaller models on large model outputs (distillation) degrades performance, but training on large model explanations improves it.
Why it matters
This research directly impacts your model distillation strategy, suggesting a shift from direct output mimicry to explanation-based learning for smaller, domain-specific models.
Hype4/10 - 2 AprResearch
Exploratory Analysis of TRLX RLHF Transformers with TransformerLens
EleutherAI Blog
EleutherAI demonstrates interpretability techniques using TransformerLens on TRLX RLHF models, exploring how they function.
Why it matters
Advancements in interpretability for RLHF models directly support G-SIB's need to understand, validate, and explain complex AI decision-making for regulatory compliance and risk management.
Hype3/10 - 25 OctResearch
A Preliminary Exploration into Factored Cognition with Language Models
EleutherAI Blog
EleutherAI research with GPT-3 shows 'factored cognition' via decomposition improves complex task performance, e.g., arithmetic.
Why it matters
Decomposition techniques can significantly improve base LLM performance on complex, multi-step tasks critical for banking operations, reducing the need for larger, costlier models.
Hype4/10 - 24 MayResearch
On the Sizes of OpenAI API Models
EleutherAI Blog
EleutherAI researchers inferred OpenAI API model sizes and architectures using performance benchmarks, revealing details about GPT-4.
Why it matters
Understanding the underlying architecture of black-box models like GPT-4 informs vendor selection and strategic dependency management by clarifying performance characteristics and potential scaling limits.
Hype4/10