Paper on low-effort language server implementations, comparing an execution-based approach with one that requires a little more effort by extracting information during parsing. Presented at the Dynamic Language Symposium, Auckland, New Zealand, December 7, 2022
Laurence Tratt: Structured Editing and Incremental Parsing
As someone who has from time to time written on parsing, and even published some parsing research, I get asked questions about parsing fairly regularly, some of which I’m even capable of answering.
Efficient and flexible incremental parsing | ACM Transactions on Programming Languages and Systems
Previously published algorithms for LR (k) incremental parsing are inefficient, unnecessarily restrictive, and in some cases incorrect. We present a simple algorithm based on parsing LR(k) sentential forms that can incrementally parse an arbitrary number ...
Introduction - Tree-sitter
Tree-sitter is a parser generator tool and an incremental parsing library. It can build a concrete syntax tree for a source file and efficiently update the syntax tree as the source file is edited. Tree-sitter aims to be:
par-team/par-lang
Par (⅋) is an experimental concurrent programming language. It's an attempt to bring the expressive power of linear logic into practice.
RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanization
This study addresses the challenge of extending Large Language Models (LLMs) to non-English languages, specifically those using non-Roman scripts. We propose an approach that utilizes the romanized form of text as an interface for LLMs, hypothesizing that its frequent informal use and shared tokens with English enhance cross-lingual alignment. Our approach involve the continual pretraining of a English LLM like Llama 2 on romanized text of non-English, non-Roman script languages, followed by instruction tuning on romanized data. The results indicate that romanized text not only reduces token fertility by 2x-4x but also matches if not outperforms native script representation across various NLU, NLG and MT tasks. Moreover, the embeddings computed on romanized text exhibit closer alignment with their English translations than those from the native script. Our approach presents a promising direction for leveraging the power of English LLMs in languages traditionally underrepresented in NLP research.
Spicy — Generating Robust Parsers for Protocols & File Formats — Spicy v1.18.0-dev.63
Spicy is a parser generator that makes it easy to create robust C++ parsers for network protocols, file formats, and more. Spicy is a bit like a “yacc for protocols”, but it’s much more than that: It’s an all-in-one system enabling developers to write attributed grammars that describe both syntax and semantics of an input format using a single, unified language. Think of Spicy as a domain-specific scripting language for all your parsing needs.
Engineering High-Performance Parsers with Data-Oriented Design
Notes from building Yuku: the AST is flat arrays of u32 indices instead of a pointer tree, and memory layout, allocation, strings, unicode, and serialization all follow from that one decision.
Crowdsourcing the evolution of text parsing with unified
unified is a tool for manipulating content using syntax trees. Today, we’re announcing the unified collective to make it sustainable
ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities
Recent large language models (LLMs) advancements sparked a growing research interest in tool assisted LLMs solving real-world challenges…
Tree-sitter - a new parsing system for programming tools - Strange Loop
Strange Loop is a conference for software developers covering programming langs, databases, distributed systems, security, machine learning, creativity, and more!
Arturo programming language
Arturo programming language: Simple, expressive & portable programming language for efficient scripting
proto - A multi-language version manager | moonrepo
Lightspeed version manager for programming languages and their dependency managers.
Kaitai Struct: declarative binary format parsing language
Kaitai Struct is a formal language for binary format specification that can be compiled into parser code
news/faster-builds
A delightful language with friendly error messages, great performance, small assets, and no runtime exceptions.
aviatesk/JETLS.jl
A new language server for Julia, enabling modern, compiler-powered tooling.
Introducing gpt-oss
We’re releasing gpt-oss-120b and gpt-oss-20b—two state-of-the-art open-weight language models that deliver strong real-world performance at low cost. Available under the flexible Apache 2.0 license, these models outperform similarly sized open models on reasoning tasks, demonstrate strong tool use capabilities, and are optimized for efficient deployment on consumer hardware.
