Wiki

LeNet-5 is a pioneering work in the field of deep learning and convolutional neural networks, which laid the foundation for many key concepts in modern deep learning, such as convolutional layers, pooling layers, and fully connected layers.

Qualification questions focus on how to determine all the conditions or factors required for an action or event to be successfully performed in a changing environment.

ReAct proposes a general paradigm that combines advances in reasoning and action to enable language models to solve a variety of language reasoning and decision-making tasks.

Pre-training Once is a three-branch self-supervised training framework that introduces elastic student branches and randomly samples sub-networks for training in each pre-training step.

FlexAttention is a flexible attention mechanism designed to improve the efficiency of high-resolution vision-language models.

FlashAttention is an efficient and memory-friendly attention algorithm.

Causal Attention (CATT) is an innovative attention mechanism that improves the interpretability and performance of models by incorporating causal inference, especially in vision-language tasks. This mechanism was first proposed by researchers from Nanyang Technological University and Monash University in Australia in 20 […]

Thought Trees generalize the popular thought chaining approach to prompt language models and enable the exploration of coherent text units (thoughts) as intermediate steps in problem solving.

The MoMa architecture is a novel modality-aware mixture of experts (MoE) architecture designed for pre-training mixed-modality, early-fusion language models.

Multi-step Error Minimization (MEM) was published in 2024 by the Institute of Information Engineering of the Chinese Academy of Sciences, Nanyang Technological University, National University of Singapore, and Sun Yat-sen University in the paper “Multimodal Unlearnable E […]

The Geometric Langlands Conjecture is a geometric version of the Langlands program.

The Langlands Program is a highly influential research field in modern mathematics. It involves multiple branches of mathematics such as number theory, algebraic geometry and group representation theory, and attempts to reveal the profound connections between them.

An application-specific integrated circuit (ASIC) is an integrated circuit designed and manufactured according to specific user requirements and the needs of a specific electronic system.

Wall clock time is a term used to measure the running time of a program or process. It refers to the actual time taken from the start of program execution to the end, including all types of waiting and blocking time.

Pareto Front is a key concept in multi-objective optimization, which refers to a set of solutions that achieve the best trade-off between multiple objectives.

Stride is a term that is often used in image processing and convolutional neural networks (CNNs). In the context of image processing, stride refers to the number of steps that the operation window moves on the image when applying certain operations to the image, such as cropping, feature extraction, or filtering. For example, when cropping an image, […]

Dynamic Prompts is a prompting technique that allows prompts to be dynamically adjusted based on specific tasks or instances in natural language processing (NLP) and other artificial intelligence applications. This technique can significantly improve the performance and adaptability of models. Dyn […]

Simple Online and Realtime Tracking (SORT) is a practical multi-target tracking method that focuses on simple and efficient algorithms. It was presented by researchers from Queensland University of Technology and the University of Sydney at the 2016 IEEE International Conference on Image Processing. […]

Prioritized Experience Replay is a method for reinforcement learning that replays experiences at different frequencies based on their importance, thereby improving learning efficiency.

CoT technology decomposes complex problems into a series of step-by-step sub-problem answers, guiding the model to generate a detailed reasoning process, thereby improving the model's performance on complex tasks such as arithmetic reasoning, common sense reasoning, and symbolic reasoning.

Parameter Efficient Fine-tuning (PERT) is a fine-tuning method for large pre-trained models that reduces computational and storage costs by fine-tuning only a small subset of model parameters while maintaining performance comparable to full-parameter fine-tuning.

In the field of artificial intelligence, a "world model" is a model that can characterize the state of the environment or the world and predict the transition between states. This model enables the agent to learn in a simulated environment and transfer the learned strategy to the real world, thereby improving learning efficiency and reducing risks. Jürgen S […]

Multimodal Contrastive Learning with Joint Example Selection (JEST) aims to address the high energy consumption problem during training of large language models such as ChatGPT.

Full Parameter Tuning is a model optimization technique in deep learning, especially used in the context of transfer learning or domain adaptation. It involves fine-tuning all parameters of a pre-trained model to adapt it to a specific task or dataset.