Repository navigation
Expand file tree
/
Copy pathsearch.json
More file actions
1 lines (1 loc) · 76.3 KB
/
Copy pathsearch.json
File metadata and controls
1 lines (1 loc) · 76.3 KB
1
[{"title":"BeyondMimic","url":"/posts/79c4b1aa.html","date":"2026-06-18","updated":"2026-06-18","tags":["Paper","Robotics","Humanoid","Motion Imitation","Diffusion Policy","Whole-Body Control"],"categories":["Paper","Robotics","Humanoid Control"],"excerpt":"BeyondMimic Type: Paper Topic: From motion tracking to versatile humanoid control with guided diffusion Links: Project / arXiv One Sentence BeyondMimic first learns strong humanoid motion tracking from human motions, then distills those mot","content":"BeyondMimic Type: Paper Topic: From motion tracking to versatile humanoid control with guided diffusion Links: Project / arXiv One Sentence BeyondMimic first learns strong humanoid motion tracking from human motions, then distills those motion skills into a diffusion policy that can be guided at test time to solve new tasks. What Problem It Solves Motion imitation is useful, but plain imitation has a ceiling: it can copy reference clips but downstream tasks often need new motion combinations and robots must react to goals, obstacles, joystick commands, or waypoints So the question is: How do we move from tracking motion clips to controlling a humanoid flexibly? Core Idea BeyondMimic separates the problem into two levels. 1. Build strong motion tracking skills The system first learns to track challenging human motions on humanoid hardware, including highly dynamic skills such as sprinting, jumping spins, and cartwheels. This stage is about making motion data physically executable. 2. Distill skills into a guided diffusion policy Instead of only replaying or tracking existing clips, the learned motion primitives are distilled into a unified diffusion policy. At test time, simple cost functions can guide the diffusion process toward a task goal. Examples include: waypoint navigation joystick teleoperation obstacle avoidance motion synthesis beyond the exact training clips Important Innovation The important jump is from imitation to composition. DeepMimic-style methods show how to learn a skill from a reference. BeyondMimic asks how to combine learned skills into new task-directed behavior without retraining for every task. The guided diffusion policy is the key mechanism: the policy carries a learned motion prior guidance pushes the generated behavior toward a downstream objective the result can remain naturalistic while being task-specific Why It Matters For humanoids, natural motion and task control are often in tension. BeyondMimic is interesting because it tries to keep both: natural whole-body motion from human demonstrations flexible control from test-time guidance This is closer to how we want humanoid robots to behave: not just copying a library of clips, but reusing motion knowledge to solve new situations. Relation To DeepMimic And OmniRetarget DeepMimic gives the basic idea: use reference motion to train physically robust skills. OmniRetarget focuses on making the reference data better, especially when interactions with objects and terrain matter. BeyondMimic focuses on what happens after imitation: how to compose motion skills into versatile control. Limitation The framework still needs a strong motion tracking pipeline and a useful motion dataset. Diffusion guidance also depends on whether the test-time cost captures the task well. If the cost is poorly designed, guidance can push the policy toward behavior that satisfies the objective but loses motion quality. Takeaway BeyondMimic is not just “humanoid imitates human motion”. Its real point is: Learn a rich motion prior, then guide it to solve new control tasks. That is why it is a useful bridge between motion imitation and general-purpose humanoid control."},{"title":"DeepMimic","url":"/posts/4d2ac08f.html","date":"2026-06-18","updated":"2026-06-18","tags":["Paper","Robotics","Motion Imitation","Reinforcement Learning","Physics-Based Control"],"categories":["Paper","Robotics","Motion Imitation"],"excerpt":"DeepMimic Type: Paper Venue: SIGGRAPH 2018 Topic: Example-guided RL for physics-based character skills Links: Project / ACM One Sentence DeepMimic teaches a simulated character to perform a motion clip by using reinforcement learning with a","content":"DeepMimic Type: Paper Venue: SIGGRAPH 2018 Topic: Example-guided RL for physics-based character skills Links: Project / ACM One Sentence DeepMimic teaches a simulated character to perform a motion clip by using reinforcement learning with an imitation reward that keeps the policy close to the reference motion while still obeying physics. What Problem It Solves Motion capture tells us what a body should look like, but it does not directly give a controller that can survive in a physical simulator. The key difficulty is: If we only replay animation, the motion is not physically robust. If we only optimize task reward, the character may find unnatural tricks. We want both: realistic motion style and physically stable control. Core Idea Train a policy with RL, but make the reward strongly reference-aware. At each time step, the policy is rewarded for matching the reference motion in several ways: pose similarity velocity similarity end-effector position similarity center-of-mass behavior So the agent is not just asked to “move forward” or “jump”. It is asked to solve the task in the style of the demonstration. Pipeline Prepare one or more reference motion clips. Simulate a physics-based character. Train a policy with imitation reward plus task reward. The learned policy can recover from perturbations better than pure playback because it has learned feedback control. Important Innovation The important contribution is not simply “use RL to imitate motion”. The important part is that DeepMimic gives a practical reward design for turning motion clips into robust motor skills. This makes example motions usable as training signals for physics-based control. Why It Matters DeepMimic is one of the conceptual roots behind many modern humanoid motion imitation papers. The pattern is still everywhere: collect or generate reference motions retarget them to a robot or character train a policy to track them use the learned controller as a reusable skill prior Limitation DeepMimic is still mostly about tracking or reproducing demonstrated skills. It does not by itself solve the harder question: How do we compose many motion skills to solve new downstream tasks? That is exactly where later work such as BeyondMimic becomes interesting. Takeaway DeepMimic turns motion imitation into a reinforcement learning control problem. The central lesson is simple: a good imitation reward can convert reference clips into physically robust skills."},{"title":"OmniRetarget","url":"/posts/9e6b2c11.html","date":"2026-06-18","updated":"2026-06-18","tags":["Paper","Robotics","Humanoid","Motion Retargeting","Loco-Manipulation","Data Generation"],"categories":["Paper","Robotics","Motion Retargeting"],"excerpt":"OmniRetarget Type: Paper Topic: Interaction-preserving data generation for humanoid whole-body loco-manipulation Links: Project / arXiv / Dataset One Sentence OmniRetarget converts human motions into robot trajectories while preserving impo","content":"OmniRetarget Type: Paper Topic: Interaction-preserving data generation for humanoid whole-body loco-manipulation Links: Project / arXiv / Dataset One Sentence OmniRetarget converts human motions into robot trajectories while preserving important contacts and spatial relations with objects, terrain, and the environment. What Problem It Solves Humanoid skill learning often starts from human motion data. But direct retargeting is messy because humans and robots have different bodies. Common failure cases: foot skating body or object penetration broken hand-object contact plausible-looking motion that is useless for robot training For loco-manipulation, preserving interaction is more important than simply matching joint angles. Core Idea Represent interaction structure explicitly. Instead of only retargeting body pose, OmniRetarget models the spatial and contact relationships between: the agent terrain manipulated objects environment geometry The paper uses an interaction mesh to preserve these task-relevant relationships while adapting the motion to the humanoid embodiment. Pipeline Start from human motion demonstrations. Build an interaction representation between human, object, terrain, and scene. Retarget the motion to the humanoid while enforcing kinematic feasibility. Generate trajectories that can be used as references for RL policy training. Augment one demonstration across different objects, terrains, and configurations. Important Innovation The key innovation is that retargeting is not treated as a body-only pose matching problem. OmniRetarget says: The contact relation is part of the motion. For humanoid loco-manipulation, this is critical. A pushing, carrying, climbing, or parkour motion is defined not only by limb positions, but also by how the body interacts with the world. Why It Matters High-quality reference data can make RL much simpler. The paper reports that high-quality retargeted trajectories allow policies to learn long-horizon skills on a Unitree G1 humanoid with a relatively simple reward setup, instead of requiring heavy curricula or many task-specific tricks. What To Remember OmniRetarget is best understood as a data engine. It is not mainly proposing a new policy architecture. It improves the input side of humanoid learning: better references fewer physical artifacts interaction-aware augmentation more useful data for whole-body RL Limitation The method still depends on the quality and coverage of source demonstrations and on how well the interaction representation captures the actual task. If the demonstration misses an important contact mode, retargeting alone cannot invent the skill. Takeaway OmniRetarget is about preserving what matters during retargeting: contacts, spatial relations, and task interaction. For humanoid learning, this can be more important than matching human pose perfectly."},{"title":"Docker Setup","url":"/posts/d0c4e12.html","date":"2026-06-18","updated":"2026-06-18","tags":["Linux","Docker","ROS","Setup"],"categories":["Linux","System"],"excerpt":"Docker setup notes split out from Linux Lookup . Install Docker On Ubuntu 22.04 # remove old versions sudo apt-get remove docker docker-engine docker.io containerd runc # install docker sudo apt-get update sudo apt-get upgrade sudo apt-get ","content":"Docker setup notes split out from Linux Lookup . Install Docker On Ubuntu 22.04 # remove old versions sudo apt-get remove docker docker-engine docker.io containerd runc # install docker sudo apt-get update sudo apt-get upgrade sudo apt-get install -y docker.io # allow current user to run docker sudo usermod -aG docker \" $USER \" newgrp docker # verify docker --version docker run --rm hello-world Docker Daemon Proxy If pulling images is blocked by proxy or network settings, configure a Docker daemon proxy. sudo mkdir -p /etc/systemd/system/docker.service.d sudo tee /etc/systemd/system/docker.service.d/http-proxy.conf > /dev/null 'EOF' [Service] Environment=\"HTTP_PROXY=http://proxy-host:port\" Environment=\"HTTPS_PROXY=http://proxy-host:port\" Environment=\"NO_PROXY=localhost,127.0.0.1,::1\" EOF sudo systemctl daemon-reload sudo systemctl restart docker sudo systemctl show --property = Environment docker # verify again docker run --rm hello-world ROS Jazzy Container Pull the image: docker pull osrf/ros:jazzy-desktop-full Create the project container: docker run -it --name mtc_jazzy \\ --network host \\ --ipc host \\ -v \" $HOME /Robotic:/home/haotian/Robotic\" \\ -w /home/haotian/Robotic \\ -e http_proxy = \" $http_proxy \" \\ -e https_proxy = \" $https_proxy \" \\ -e HTTP_PROXY = \" $HTTP_PROXY \" \\ -e HTTPS_PROXY = \" $HTTPS_PROXY \" \\ -e NO_PROXY = \" $NO_PROXY \" \\ -e no_proxy = \" $no_proxy \" \\ osrf/ros:jazzy-desktop-full \\ bash Enter the same container later: docker start -ai mtc_jazzy If the container is already running, open another shell inside it: docker exec -it mtc_jazzy bash If group permissions have not refreshed in the current shell: newgrp docker Daily Commands docker ps # running containers docker ps -a # all containers docker stop mtc_jazzy # stop container docker rm mtc_jazzy # remove container, not host code docker images # local images docker rmi image > # remove image docker run --rm -it IMAGE_ID > /bin/bash"},{"title":"Linux Lookup","url":"/posts/f7b0b433.html","date":"2025-08-20","updated":"2026-06-18","tags":["Linux","Command","Shell","Git","CUDA","Python"],"categories":["Linux","System"],"excerpt":"This page is a personal Linux command lookup sheet. It now also includes the older Linux blog notes. Links Ubuntu 20.04 setup notes Vim Handbook Docker Setup Shell Basics Basic Commands man : show the manual page for a command. mv : move a ","content":"This page is a personal Linux command lookup sheet. It now also includes the older Linux blog notes. Links Ubuntu 20.04 setup notes Vim Handbook Docker Setup Shell Basics Basic Commands man : show the manual page for a command. mv : move a file or directory. rm : remove a file or directory. cat : print file content. fg : resume a paused process in the foreground. bg : continue a paused process in the background. jobs : list jobs started from the current shell. touch : create an empty file or update its timestamp. chmod +x : make a file executable. Shell Tips Use * to match multiple characters and ? to match one character. Ctrl+D : exit the shell. Ctrl+Z : suspend the current foreground process. echo \"content\" > file : overwrite a file. echo \"content\" >> file : append to a file. grep \"pattern\" file : search for a pattern. sed -i \"/pattern/d\" file : delete matching lines in place. Nano Ctrl+X : exit. Ctrl+G : open help. Ctrl+K : cut a line. Ctrl+U : paste a line. Ctrl+W : search. Ctrl+Y / Ctrl+V : previous / next page. Ctrl+A / Ctrl+E : line start / line end. Shell Scripting # variables echo \" $VAR \" expr \" $VAR1 \" + \" $VAR2 \" read VAR # conditional if [ condition ] ; then echo \"true\" else echo \"false\" fi # loop for VAR in $LIST ; do echo \" $VAR \" done Installation Conda conda create -n myenv python = 3.8 # create a new env conda remove -n myenv --all # delete the whole env conda create -n new_env --clone old_env conda env update -f requirement.yml PyTorch, Torchvision, Python NVIDIA driver: open Software & Updates , then use the Additional Drivers tab. Install torch / torchvision / torchaudio with a specific CUDA wheel index: pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu128 Install a local wheel: pip install package.whl Temporary PyPI mirror: pip install package_name -i https://pypi.tuna.tsinghua.edu.cn/simple Global PyPI mirror: pip config set global.index-url https://mirrors.cloud.tencent.com/pypi/simple Venv python3 -m venv venv source venv/bin/activate deactivate AppImage chmod a+x *.AppImage ./*.AppImage Desktop Shortcut Create a .desktop file under ~/.local/share/applications/ : [ Desktop Entry ] Name = Another Redis Desktop Comment = Another Redis Desktop Exec = /opt/Another_Redis_Desktop/Another-Redis-Desktop-Manager.1.5.6.AppImage Icon = /opt/Another_Redis_Desktop/ard.png Terminal = false Type = Application Categories = Developer; App Notes Zotero: run ./set_launcher_icon , then link the generated desktop file: ln -s /path/to/zotero/zotero.desktop ~/.local/share/applications/zotero.desktop Zotero WebDAV can use Jianguoyun: https://dav.jianguoyun.com/dav/ . QQ Music sandbox workaround: qqmusic --no-sandbox System & Terminal Supervise GPU usage: watch -n 10 nvidia-smi Look for proxy: env | grep -i proxy Check system architecture: uname -m Install / uninstall a .deb package: sudo dpkg -i package_name.deb sudo dpkg -r package_name Show Python interpreter path: import sys path = sys . executable print ( path ) Set proxy in ~/.bashrc or ~/.zshrc , then source ~/.bashrc or source ~/.zshrc : export https_proxy = http://proxy-host:port export http_proxy = http://proxy-host:port Unpack Files cat event.tar.gz.part-* > event.tar.gz tar -xvf archive.tar tar -xzvf archive.tar.gz tar -xjvf archive.tar.bz2 unzip archive.zip -d /path/to/directory unrar e archive.rar Download From URL aria2c -s 10 -x 10 \"URL\" -s is the split count, and -x is the number of connections per server. Install / Uninstall Apps sudo apt remove package_name > # remove app only sudo apt purge package_name > # remove app and configuration files Terminal Sessions Nohup Run a script in the background: nohup /root/runoob.sh & The output is written to nohup.out by default. Find and kill the process: ps -aux | grep \"runoob.sh\" kill -9 PID > Tmux vim ~/.tmux.conf tmux source-file ~/.tmux.conf tmux new -t session_name tmux ls tmux kill-window -t session_name Detach window: Ctrl+B , then D . Zellij ctrl-p + d : create a pane below. ctrl-p + r : create a pane on the right. ctrl-p + x : close current pane. ctrl-q : quit Zellij. Python python -c \"print('Hello from command line')\" python -c \"import torch; print(torch.cuda.is_available()); print(torch.__version__)\" python -m pip install package_name python main.py -h Docker Docker installation and ROS container setup moved to Docker Setup . Common commands: docker images docker rmi REPOSITORY:TAG > docker run --rm -it IMAGE_ID > /bin/bash docker ps docker ps -a Git Basic Usage git status git add filename.txt git add . git commit -m \"commit message\" git commit -am \"commit message\" git push -u origin new_branch Failure With Git Connection If port 22 is blocked, use SSH over port 443: text Host github.com Hostname ssh.github.com Port 443 User git Put the block in ~/.ssh/config . Set Username And Email git config --global user.name \"Tesla-SHT\" git config --global user.email \"you@example.com\" For private repositories, use a username and PAT instead of a password. Remote & Local Branch Pull a remote branch to local: git checkout -b branch origin/branch Delete a local branch: git checkout another_branch git branch -D branch Checkout, Switch, Restore checkout can both move branches and restore files. switch and restore split those ideas more clearly. # checkout git checkout develop git checkout -b new-feature git checkout commit-hash > git checkout -- a.txt git checkout HEAD -- a.txt # switch git switch develop git switch -c new-feature git switch - # restore git restore a.txt git restore --staged a.txt git restore --source = HEAD a.txt Merge One Branch To Another git pull --rebase : fetch remote commits and replay local commits after them. git push : push current branch. git checkout dev : switch to dev . git merge main : merge main into dev . Modify The Latest Commit git reset --soft HEAD^ git reset --hard HEAD~1 git commit --amend Use git reset --hard only when you intentionally want to discard local changes. Submodule Tracking Sometimes a submodule may not be tracked as expected. Remove the original tracking: git rm --cached file-or-dir > Delete the .git file in the submodule and delete the related file under the root .git directory if it exists. git add the file again. CUDA Assign Process To GPU CUDA_VISIBLE_DEVICES = 0 python main.py export CUDA_VISIBLE_DEVICES = 0 import os os . environ [ \"CUDA_VISIBLE_DEVICES\" ] = \"0\" import torch torch . cuda . is_available ( ) torch . cuda . device_count ( ) torch . cuda . get_device_name ( 0 ) torch . cuda . current_device ( ) VSCode Python Interpreter Ctrl+Shift+P : open command palette. Choose Python: Select Interpreter . Shortcuts Ctrl+K Ctrl+S : open keyboard shortcuts. Ctrl+Shift+F : global search. Ctrl+P : quick open. Ctrl+Shift+G : source control. Ctrl+Shift+D : debug. Ctrl+K M : select language mode. Ctrl+Alt+N : run code with Code Runner. Alt+Z : toggle word wrap. Ctrl+Shift+P : command palette. Regular Expression Table Basic Syntax Quantifiers Position Group Common Usage Any digit: \\d Zero or more: * Start of line: ^ (abc) group, use $1 in replace Empty line: ^\\s*$ Word character: \\w One or more: + End of line: $ [abc] matches one of a , b , c Comments: //.* Whitespace: \\s Zero or one: ? Word boundary: \\b a|b matches a or b Quoted text: \"[^\"]*\" Any character: . Exact count: {n} Escape regex metacharacters such as dot, star, plus, question mark, brackets, pipe, and backslash. Add SiliconFlow Model Install an OAI-compatible extension. Set the OAI base URL to https://api.siliconflow.cn/v1 . Add an OAI model in Copilot or Cline. Desktop Environment Xfce If X server already running on display :0 appears, run: exec startxfce4 Modify lockscreen time: Section \"ServerFlags\" Option \"BlankTime\" \"0\" Option \"StandbyTime\" \"0\" Option \"SuspendTime\" \"0\" Option \"OffTime\" \"0\" EndSection Errors Some Text Is Missing In GNOME Restart GNOME Shell: Press Alt+F2 . Type r . Press Enter . dpkg: error: dpkg frontend lock is locked by another process ps -e | grep apt sudo kill PID >"},{"title":"Linux","url":"/posts/15691.html","date":"2025-08-19","updated":"2026-06-18","tags":["Linux","Command","Shell"],"categories":["Linux","Notes"],"excerpt":"This note has been merged into Linux Lookup . The original shell basics, editor usage, and shell scripting snippets now live in that lookup page so the Linux notes stay in one place.","content":"This note has been merged into Linux Lookup . The original shell basics, editor usage, and shell scripting snippets now live in that lookup page so the Linux notes stay in one place."},{"title":"ED-SLAM","url":"/posts/67f0b1ee.html","date":"2025-09-24","updated":"2026-06-18","tags":["Paper","Computer Vision","3DGS","Gaussian Splatting","SLAM","Event Camera"],"categories":["Paper","Computer Vision","SLAM"],"excerpt":"ED-SLAM Type: Paper Venue: ICRA 2026 Topic: Event-depth SLAM + Gaussian Splatting Motivation 3DGS-SLAM can build dense and high-fidelity maps, but most systems still depend heavily on RGB or RGB-D tracking. Conventional RGB tracking is frag","content":"ED-SLAM Type: Paper Venue: ICRA 2026 Topic: Event-depth SLAM + Gaussian Splatting Motivation 3DGS-SLAM can build dense and high-fidelity maps, but most systems still depend heavily on RGB or RGB-D tracking. Conventional RGB tracking is fragile under fast motion, motion blur, low light, or textureless scenes. Existing event-based GS methods often assume known poses or become unstable when processing long event streams. ED-SLAM tries to make event-depth SLAM more practical: no ground-truth camera poses, robust tracking, and 3DGS-based dense mapping. Main Idea ED-SLAM separates the system into two coupled parts: Tracking: convert incoming events into time-surface maps, then estimate pose with depth-aware patch alignment. Mapping: use the estimated continuous trajectory and raw event stream to optimize a 3D Gaussian map. The key design is that events are not only an auxiliary signal. They are used for both robust front-end tracking and fine-grained mapping. Input & Output Input Time-aligned event stream Depth images Camera intrinsics Output 6-DoF camera trajectory 3D Gaussian scene representation Renderable dense reconstruction Pipeline Time Surface Map During tracking, raw events are aggregated into a time-surface map (TSM). Each pixel stores a decayed timestamp of recent events: T ( x , t ) = exp ( − t − t l a s t ( x ) τ ) T(x,t)=\\exp\\left(-\\frac{t-t_{last}(x)}{\\tau}\\right) T ( x , t ) = exp ( − τ t − t l a s t ( x ) ) The intuition is simple: event cameras naturally fire around image edges, so TSM gives a sharp and low-latency edge-like representation. This is much more stable than RGB images when the camera moves quickly or illumination is poor. Patch-based Event-Depth Tracking Instead of aligning the whole image globally, ED-SLAM samples local patches from high-response TSM regions. For each patch, depth provides the geometry needed for reprojection: P i ′ = K T K − 1 P i P'_i = KTK^{-1}P_i P i ′ = K T K − 1 P i Then the warped patch is sampled on the paired TSM by bilinear interpolation: t ^ i t a r = S b i l i n e a r ( T S M s r c , P i ′ ) \\hat{t}^{tar}_i=S_{bilinear}(TSM_{src}, P'_i) t ^ i t a r = S bi l in e a r ( T S M sr c , P i ′ ) The tracking objective is bidirectional: target-to-source and source-to-target are optimized together. T ∗ = arg min T ∑ i ∥ t i t a r − t ^ i t a r ∥ 2 2 + ∑ j ∥ t j s r c − t ^ j s r c ∥ 2 2 T^*=\\arg\\min_T \\sum_i \\|t_i^{tar}-\\hat{t}_i^{tar}\\|_2^2+ \\sum_j \\|t_j^{src}-\\hat{t}_j^{src}\\|_2^2 T ∗ = ar g T min i ∑ ∥ t i t a r − t ^ i t a r ∥ 2 2 + j ∑ ∥ t j sr c − t ^ j sr c ∥ 2 2 This bidirectional formulation is important because one-way alignment can be biased by occlusion, missing events, or noisy local regions. Mapping 3D Gaussian Representation The scene is represented as a set of 3D Gaussians. Each Gaussian stores: mean position covariance / scale / rotation opacity color Rendering follows the standard differentiable 3DGS rasterization pipeline, so both image-like signals and event signals can provide gradients. Continuous Trajectory Model Events are asynchronous, so a single pose per frame is too coarse. ED-SLAM interpolates camera poses on the SE(3) manifold: T ( t k ) = T s t a r t ⋅ exp ( t k − t s t a r t t e n d − t s t a r t log ( T s t a r t − 1 T e n d ) ) T(t_k)=T_{start}\\cdot \\exp\\left(\\frac{t_k-t_{start}}{t_{end}-t_{start}} \\log(T_{start}^{-1}T_{end})\\right) T ( t k ) = T s t a r t ⋅ exp ( t e n d − t s t a r t t k − t s t a r t lo g ( T s t a r t − 1 T e n d ) ) This allows the system to render brightness changes between close timestamps and compare them with real event measurements. Event Loss The rendered event signal is the log-brightness difference between two rendered frames: E ^ ( x ) = log ( I ^ k + Δ t ( x ) ) − log ( I ^ k ( x ) ) \\hat{E}(x)=\\log(\\hat{I}_{k+\\Delta t}(x))-\\log(\\hat{I}_{k}(x)) E ^ ( x ) = lo g ( I ^ k + Δ t ( x )) − lo g ( I ^ k ( x )) The real event signal is accumulated from raw events during the same time interval: E ( x ) = C { e i ( x , t i , p i ) } t k t i t k + Δ t E(x)=C\\{e_i(x,t_i,p_i)\\}_{t_k E ( x ) = C { e i ( x , t i , p i ) } t k t i t k + Δ t The mapping loss directly compares these two signals: L e v e n t = ∥ E ( x ) − E ^ ( x ) ∥ 2 L_{event}=\\|E(x)-\\hat{E}(x)\\|_2 L e v e n t = ∥ E ( x ) − E ^ ( x ) ∥ 2 Why It Works TSM converts sparse asynchronous events into a trackable edge representation. Depth makes patch warping geometrically meaningful, avoiding pure 2D photometric matching. Bidirectional patch alignment improves robustness under partial visibility and noisy events. Continuous trajectory modeling matches the temporal nature of events. 3DGS provides a compact and differentiable scene representation for dense mapping. Experiments The paper evaluates ED-SLAM on both synthetic and real-world event datasets. The main observed improvements are: better tracking stability over long trajectories lower pose drift than previous event-based SLAM / GS methods higher-quality reconstruction under fast motion and difficult lighting Takeaways ED-SLAM is interesting because it does not simply plug events into an RGB-D SLAM system. It builds a dedicated event-depth front end and connects it with 3DGS mapping. For me, the most important part is the patch-based bidirectional tracker: it is small compared with the whole system, but it decides whether the downstream Gaussian map can remain stable."},{"title":"Event3R","url":"/posts/e3a7263f.html","date":"2026-06-18","updated":"2026-06-18","tags":["Paper","Computer Vision","3D Reconstruction","Vision Foundation Model","Event Camera"],"categories":["Paper","Computer Vision","3D Reconstruction"],"excerpt":"Event3R Type: Paper Venue: IROS 2026 Topic: Event-based feed-forward 3D reconstruction Motivation DUSt3R-style methods show that feed-forward dense 3D reconstruction from images can be fast and globally consistent. However, RGB frames are s","content":"Event3R Type: Paper Venue: IROS 2026 Topic: Event-based feed-forward 3D reconstruction Motivation DUSt3R-style methods show that feed-forward dense 3D reconstruction from images can be fast and globally consistent. However, RGB frames are still vulnerable to fast motion, motion blur, high dynamic range scenes, and low light. Event cameras are naturally robust in these conditions, but their data is asynchronous, sparse, and hard to feed into image-based reconstruction backbones. Event3R asks a direct question: can we reconstruct a globally aligned 3D pointmap from pure event streams in one feed-forward pass? Input & Output Input Two or more event stream segments Each segment is converted into a spatial-temporal voxel representation Output Globally aligned 3D pointmaps Dense depth / pose-related geometry from event-only input Pipeline Event3R has three main stages: Event-to-voxel representation Temporal feature aggregation DUSt3R-style spatial reconstruction and decoding The core is to bridge two worlds: event stream: asynchronous and temporally dense feed-forward 3D reconstruction: usually expects image-like synchronized inputs Event-to-Voxel Representation Each event is represented as: e k = ( u k , t k , p k ) e_k=(u_k,t_k,p_k) e k = ( u k , t k , p k ) where u k = ( x k , y k ) u_k=(x_k,y_k) u k = ( x k , y k ) is the pixel location, t k t_k t k is the timestamp, and p k p_k p k is the polarity. The event stream within a short temporal window is discretized into a voxel grid: E ( x , y , n ) = ∑ k p k max ( 0 , 1 − ∣ n − t k ∗ ∣ ) E(x,y,n)=\\sum_k p_k\\max(0,1-|n-t_k^*|) E ( x , y , n ) = k ∑ p k max ( 0 , 1 − ∣ n − t k ∗ ∣ ) This keeps temporal information while making the input compatible with convolution / transformer style processing. Temporal Encoder The temporal encoder is the part I find most central. A naive solution would simply collapse all event bins into one image-like tensor. But that would throw away the ordering of motion. Event3R instead processes each temporal bin as a structured frame and then performs attention across bins. For a temporal bin: F t = C o n v E m b e d ( V t ) F_t = ConvEmbed(V_t) F t = C o n v E mb e d ( V t ) For each spatial patch position, features across temporal bins form a sequence: X i = [ F i 1 , … , F i T ] X_i=[F_i^1,\\ldots,F_i^T] X i = [ F i 1 , … , F i T ] The center-bin feature is enhanced by attending to the whole temporal sequence: F i , c e n t e r ∗ = A t t e n t i o n ( Q = F i c e n t e r , K , V = X i ) F^*_{i,center}=Attention(Q=F_{i}^{center},K,V=X_i) F i , ce n t er ∗ = A tt e n t i o n ( Q = F i ce n t er , K , V = X i ) This design is useful because the output pointmap is tied to a central temporal reference, while the surrounding bins provide motion and geometry cues. Masked Bin Modeling Event datasets with dense 3D labels are limited, so Event3R introduces Masked Bin Modeling (MBM). The idea is similar in spirit to masked modeling: hide one or more temporal bins force the temporal encoder to reconstruct the missing bin learn better temporal aggregation before or during fine-tuning This is especially suitable for event data because the temporal bins are not independent frames. Neighboring bins encode motion continuity. Reconstruction Backbone After temporal aggregation, Event3R uses a DUSt3R-style reconstruction backbone: spatial encoder extracts image-like geometric features decoder exchanges information across views with cross-attention final prediction is a globally consistent 3D pointmap Compared with optimization-based event reconstruction, this feed-forward setting is much faster because it avoids per-scene iterative fitting. Training Objectives Event3R combines several objectives: supervised 3D reconstruction loss when labels are available MBM loss for temporal representation learning contrastive alignment to strengthen cross-view correspondence temporal consistency regularization to reduce unstable geometry across event segments The important design point is that temporal learning is not only a preprocessing trick. It remains part of the reconstruction training objective. Why It Works Event voxels convert asynchronous events into a learnable representation. Temporal attention keeps the motion structure instead of flattening time. MBM improves temporal feature learning when labeled event-depth-pose data is scarce. DUSt3R-style decoding gives globally aligned geometry without per-scene optimization. Experiments The paper evaluates Event3R on synthetic and real-world event benchmarks, including: dense 3D reconstruction depth estimation pose estimation challenging HDR / low-light conditions The key result is that Event3R can reconstruct robust geometry from event-only input and generalize better under conditions where RGB-based reconstruction becomes unreliable. Takeaways Event3R is a step from local event reconstruction toward global event-based 3D reconstruction. The most important idea is not only “use events instead of RGB”, but how to make asynchronous temporal information understandable for a feed-forward 3D backbone. For embodied perception and robotics, this is a useful direction: fast global geometry from event cameras could be valuable when robots move quickly or operate in difficult lighting."},{"title":"Register Token","url":"/posts/a4caa571.html","date":"2025-08-19","updated":"2025-08-23","tags":["Paper","AI","Vision Transformer","Token","Transformer"],"categories":["Paper","AI","Transformer"],"excerpt":"Background Modern vision transformers (ViTs) have a common problem that their tokens may have artifacts, which can lead to suboptimal performance. Only DINO will not have artifacts, which can be a comparison for other methods. Why and When ","content":"Background Modern vision transformers (ViTs) have a common problem that their tokens may have artifacts, which can lead to suboptimal performance. Only DINO will not have artifacts, which can be a comparison for other methods. Why and When Artifacts Occur Relation with High Norms The author finds that the artifacts are related to the high norms of tokens. Appearance in Middle of Training The high norm of tokens appears in the middle of training Information Redundancy The author finds that the artifacts are caused by information redundancy, which means if the patch is very similar to its neighbors, it is more possible to have artifacts. Conclusion Enough large and well-trained models will learn to recognize redundant tokens and use them to store, process, and extract global information. So the author proposes Register Token, which is a token that is used to store and process global information."},{"title":"3DGS","url":"/posts/f91374c3.html","date":"2025-02-10","updated":"2025-08-19","tags":["Paper","Computer Vision","3D Reconstruction","3DGS","Gaussian Splatting"],"categories":["Paper","Computer Vision","3D Reconstruction"],"excerpt":"Intro A new method for real-time radiance field rendering, which generates high-quality new views of a scene from an image set. Applying 3D Gaussian Distribution to present the radiance field realizes the fast training and real-time renderi","content":"Intro A new method for real-time radiance field rendering, which generates high-quality new views of a scene from an image set. Applying 3D Gaussian Distribution to present the radiance field realizes the fast training and real-time rendering. Comparison Feature 3DGS NeRF Mesh Representation Gaussian distribution (Hybrid) Neural field (implicit) Vertices, edges, faces (explicit) Rendering Speed Fast (Real-time) Slow Medium Memory Consumption Low High Low Detail Good Excellent Moderate Flexibility High (Differentiable) High (Differentiable) Low (hard for gradient) Application Dynamic scenes, real-time applications Static scenes, high-quality rendering Simple geometries, traditional graphics Terminology Radiance Field : A function represents the light intensity in 3D space. Inplicit radiance field utilizes neural networks (continous func) to represent the appearance and geometry Explicit radiance field uses discrete data structures (Mesh, point cloud, voxel) directly 3D Gaussian ellipsoid is distributed discretely while continuous in internal space Projection & Rasterization : Projection: Project the 3D Gaussian ellipsoids to the 2D image plane, to specify their positions and sizes. Rasterization: Transform the 2D Gaussian projection to discrete pixels and calculate the color&transparancy. Method Initialization SfM Motion Structure Recovery: Starting from the acquired series of images of the target Inference the position of each camera Get the feature points in the photos Obtain a sparse 3D point cloud in world coordinates as the initial input of 3D Gaussian distribution. 3D Gaussians Generate 3D Gaussian ellipsoids for each point in the point cloud. All parameters are learnable and optimized by backpropagation. Position : (x,y,z) Shape : Σ = R S S T R T \\Sigma = RSS^TR^T Σ = R S S T R T , where R R R is the rotation matrix and S S S is scale matrix. Color : Apply forth-order Spherical Harmonics (SH) to represent the rgb color(3*16 parameters). Opacity : α \\alpha α is the Opacity of the Gaussian ellipsoid. Rendering Projection : Splatting the 3D Gaussians to 2D image plane. Rasterization : Apply the rendering equation from opacity and color. In NeRF, C = ∑ i = 1 N α i C i T i C = \\sum_{i=1}^{N} \\alpha_i C_i T_i C = ∑ i = 1 N α i C i T i , where T i = e x p ( − ∫ t n t f σ ( r ( s ) ) ) T_i=exp(-\\int_{t_n}^{t_f} \\sigma(r(s))) T i = e x p ( − ∫ t n t f σ ( r ( s ))) is the accumulated transparancy whether the ray is absorbed, σ \\sigma σ is the density function, representing the intensity that the light is absorbed, t n t_n t n and t f t_f t f are the near and far distance of the ray. In 3DGS, the color is not obtained from accumulating the point along the ray, while from the Gaussian ellipsoid. And by sorting the depth of the ellipsoids, we use Alpha Blending: C = Σ i c i α i Π 1 i − 1 ( 1 − α j ) C = \\Sigma _i c_i \\alpha _i \\Pi_1^{i-1}(1-\\alpha _j) C = Σ i c i α i Π 1 i − 1 ( 1 − α j ) , where c i c_i c i is the color, α i \\alpha _i α i is the opacity of the Gaussian ellipsoid (more transparent, more close to 0) Loss L = ( 1 − λ ) L 1 + λ L D − S S I M L=(1-\\lambda)L_1+\\lambda L_{D-SSIM} L = ( 1 − λ ) L 1 + λ L D − S S I M L1 Loss: L 1 = 1 N ∑ i = 1 N ∣ C r e n d e r e d − C g t ∣ L_1 = \\frac{1}{N} \\sum_{i=1}^{N} |C_{rendered} - C_{gt}| L 1 = N 1 ∑ i = 1 N ∣ C r e n d er e d − C g t ∣ Compare the difference in pixel level between the rendered image and the ground truth. D-SSIM Loss: L D − S S I M = 1 − D S S I M ( C r e n d e r e d , C g t ) L_{D-SSIM} = 1 - DSSIM(C_{rendered}, C_{gt}) L D − S S I M = 1 − D S S I M ( C r e n d er e d , C g t ) Compare the structural similarity between the rendered image and the ground truth, including the luminance, contrast, and structure. SfM sample points, Σ \\Sigma Σ , RGB, α \\alpha α will be optimized by backpropagation. Optimization Pruning : Set the threshold of the opacity, and remove the Gaussian ellipsoids too close to the camera. Adaptive Control of Gaussians : Over-reconstruction (gradient is large, too steep): Clone a new Gaussian ellipsoid to cover the space along the position gradient. Under-reconstruction (variance is large, too moderate): Split the Gaussian ellipsoid into two smaller ones around it. By doing this, the newly cloned Gaussian points can better adapt to the geometric features of the scene, fill in areas where details are missing, and thus improve the quality of the reconstruction. Patchification : Split the screen into 16x16 tiles For each tile, only keep the Gaussian projections overlapped with the tile (if covering more than one tiles, copy and tag it to be unique in each tile), which reduces the computation complexity."},{"title":"Berkeley Wiki","url":"/posts/7591.html","date":"2025-08-19","updated":"2025-08-19","tags":["Berkeley","Wiki","Campus","Food","Library"],"categories":["Berkeley","Wiki"],"excerpt":"Preface Hope this blog will help freshmen to get involved in the campus and live comfortably by providing some useful info about something you’d better know while not included in the official guide or orientations. To be honest, it may also","content":"Preface Hope this blog will help freshmen to get involved in the campus and live comfortably by providing some useful info about something you’d better know while not included in the official guide or orientations. To be honest, it may also like a todo list to remind me to set foot in every corner of Berkeley, so that my content will probably repeat what the Google has said or be very personal. But I do hope you can dear to step into every library and have a peaceful afternoon, or easily get a cola from the beverage machine with a line of strange names on each button. Enjoy it! If you wanna add more info, please contact me before I come back :) Library Moffit Library Link: https://www.lib.berkeley.edu/visit/moffitt Location: The usual entrance(facing the Campanile) is the 4th floor, and remember to bring your student ID card. Time: At the most time, the 4th and 5th floor are open. 4th floor: Noise study area; 5th floor: Silent study area Tips: I think this library is the most popular one so that sometimes I can only work on a sofa for a while. You can reserve a study space (4 or 8), sit on the sofa or the desk, or even lie on the beanbag. You can drink water near the restroom, or buy the beverage outside the lib on the 4th floor. Bioscience, Natural Resources & Public Health Library Link: https://www.lib.berkeley.edu/visit/bioscience Location: The library is located in the Valley Life Sciences Building, behind a dinasour skeleton. Just get into and turn left, you will see a group of computers and the seats. Time: Usually, the library is open from 9:00 to 21:00 Tips: The lib is not that big, but the students are not that many. So it’s a good place to study. It’s the most west library on the campus (maybe in northwest, there are also some spaces around Brown’s). Every seat has a charging port and a light, just like the lib in Westlake. Dining Meal Plan There are many meal plans for students to choose. Roughly speaking, the swipe and flex dollar plans. The swipe plan is like a buffet, you can swipe your card to get a meal in the dining hall. The flex dollar plan is like a debit card, you can use it to buy food or snacks in restaurants. I bought the meal plan with 5 swipes, and I think it’s enough for me. Dining Commons (Large Dining Halls) All these meals are self-service, so you can eat as much as you can. So it’s a good choice to have brunch in the dining hall. Meal Price Breakfast $10 Lunch $11 Dinner $12 Crossroads Location: Northeast of the campus (a little east from Cafe 3) The biggest dining hall on the campus, with a variety of food and drinks, including salad, pizza, rice, noodels, dessert and burgers. And it will change the menu everyday. Cafe 3 location: South of the campus The dining hall is smaller than Crossroads, but the food is also delicious, like rice, macaroni and salad. Foothill location: Northeast of the campus Similiar to Cafe 3, but the dining hall is more beautiful and wide. But it’s on the hill, so you need to climb up to get there. I always want to have a drink of Berkeley cola, but I can only find it in Footill. However, the cola has no bubbles… Clark Kerr location: Too far away from the main campus… Campus Restaurants (Small Dining stores) The price is 17$ for a meal even if you have a student card, and the food is ordered by the menu, so using swipes is a better choice. Golden Bear Cafe location: Near the south gate of the campus The best choice to have a quick meal due to its location and food type: Burger, salad and drinks. Brown’s location: Northwest of the campus The food is composed of rice/pea, chicken/salmon, a side and a drink. The taste is good. But DO NOT go there after a class, you will line up around the whole hallway. 杂感 一座几乎由这所大学构成的城市,散布着忙碌着背包的师生,或是慵懒着漫步的老人,亦或是亢奋着呼号的流浪汉。一条主街道,一块美食区,剩下的便是宿舍、图书馆和教室。似乎到处都是一股纯粹的氛围,让我忘记身处何地,倒是时常弥漫的奇怪的味道,告诉我这里是美国加州的一个城市。校园和城市没有明显的界限,但是走进校园的那一刻,一切似乎变得更加随性、自由:有在草地上日光浴的,有在南门街上摆摊的,有扛着大号蹒跚的,有为巴勒斯坦呐喊的,当然更多的是急匆匆从图书馆或是教学楼里出来,奔向下一堂课的。 海岸拨开层叠的房屋,落日于晚霞中弥散;细雨浸湿了羽绒,狂风被秋叶吹乱; 钟声从隐约中传来,山腰缭绕着白雾;铃铛摇曳在音乐中,节日在灯火里阑珊。"},{"title":"DUSt3R","url":"/posts/2eb69c4a.html","date":"2025-08-01","updated":"2025-08-19","tags":["Paper","Computer Vision","3D Reconstruction","Vision Foundation Model"],"categories":["Paper","Computer Vision","3D Reconstruction"],"excerpt":"DUSt3R Type: Paper Notebook: Paper ( https://www.notion.so/Paper-17de7e7bfd4c80e2bba1f0fe3a6c1131?pvs=21 ) Motivation The traditional SfM task (estimating the position of a 3D point given a sparse set of correspondences of multiple images a","content":"DUSt3R Type: Paper Notebook: Paper ( https://www.notion.so/Paper-17de7e7bfd4c80e2bba1f0fe3a6c1131?pvs=21 ) Motivation The traditional SfM task (estimating the position of a 3D point given a sparse set of correspondences of multiple images and their image features) will divide it into sub-tasks, including the parameterization, matching, feature. But the previous sub-tasks will feed the error into the next sub-task So DUSt3R construct the network model end-to-end Input & Output Two images from two views PointMap (HxWx3) From the camera to the object, the position (x,y) records the 3D coordinate of the closest object, while it will be sheltered by the Translucent object ConfidenceMap (HxW) The true probability of each point in Pointmap Network Put the image into ViT encoder with shared weights to generate two tokens F1 and F2 The Transformer Decoder will perform self-attention and then exchange info via cross-attention Output pointmaps and confidence maps. Loss Function 3D Regression Loss L r e g r = ∣ ∣ 1 z X i ˉ j − 1 z X i j ∣ ∣ L_{regr}=||\\frac 1 z \\bar{X_i}^j-\\frac 1 z X_i^j|| L r e g r = ∣∣ z 1 X i ˉ j − z 1 X i j ∣∣ 1 z \\frac 1 z z 1 is the normalization factor, representing the average distance of the 3D points from the origin It is the 3D distance error between the true points and PointMap points Confidence-aware Loss L c o n f = ∑ v = 1 , 2 ∑ i ∈ D v C i v , 1 l r e g r ( v , i ) − α log C i v , 1 L_{conf}=\\sum_{v=1,2} \\sum_{i\\in D^v} C_i^{v,1}l_{regr}(v,i)-\\alpha \\log C_i^{v,1} L co n f = v = 1 , 2 ∑ i ∈ D v ∑ C i v , 1 l r e g r ( v , i ) − α lo g C i v , 1 Multiplication of the confidence value and the 3D distance error, so that the confidence with larger distance will be lowered"},{"title":"EGS-SLAM","url":"/posts/778f5e7a.html","date":"2025-08-19","updated":"2025-08-19","tags":["Paper","Computer Vision","3DGS","Gaussian Splatting","SLAM","Event Camera"],"categories":["Paper","Computer Vision","SLAM"],"excerpt":"Background Feature-based (ORB-SLAM): Sensitive to motion blur Weak texture reconstruction Deblur GS/NeRF: offline method + image-only GS-SLAM: Vulnerable to intense motion blur Event-based (IncEventGS, E-NeRF): Offline method due to SfM EGS","content":"Background Feature-based (ORB-SLAM): Sensitive to motion blur Weak texture reconstruction Deblur GS/NeRF: offline method + image-only GS-SLAM: Vulnerable to intense motion blur Event-based (IncEventGS, E-NeRF): Offline method due to SfM EGS-SLAM: Online, anti-blurred SLAM system Model The first part is 3D Gaussian map representation, where each scene is represented by Gaussian points (including position, covariance matrix, opacity, and color parameters). When a new scene is input, it continuously reduces the photometric, event, and depth loss for rendering, thereby optimizing the start and end poses of exposure. Once the pose optimization converges in tracking, it checks whether to select it as a keyframe and updates the entire keyframe queue. After generating a new frame, it initializes some new 3D Gaussian point clouds and optimizes them jointly with various losses. Blur-Aware Tracking Camera Trajectory T ( η ) = [ S l e r p ( R 0 , R τ , η τ ) ( 1 − η τ ) t 0 + η τ t τ 0 1 ] T(\\eta)= \\begin{bmatrix} \\mathrm{Slerp}(R_0,R_\\tau,\\frac{\\eta}{\\tau}) & (1-\\frac{\\eta}{\\tau})\\mathbf{t}_0+\\frac{\\eta}{\\tau}\\mathbf{t}_\\tau \\\\ 0 & 1 \\end{bmatrix} T ( η ) = [ Slerp ( R 0 , R τ , τ η ) 0 ( 1 − τ η ) t 0 + τ η t τ 1 ] Propose a temporal interpolation model for intra-exposure trajectory since we need precise camera pose for each event. Photometric Loss Blurred Image Blurred image = Integral of sharp images during exposure time I ~ ( u ) = ∫ 0 τ I ( T ( η ) , u ) d η ≈ 1 K ∑ k = 0 K − 1 I ( T ( η k ) , u ) \\tilde{\\boldsymbol{I }}(\\boldsymbol{u})=\\int_0^\\tau\\mathcal{I}(\\mathbf{T}(\\eta),\\boldsymbol{u})d\\eta\\approx\\frac{1}{K}\\sum_{k=0}^{K-1}\\mathcal{I}(\\mathbf{T}(\\eta_k),\\boldsymbol{u}) I ~ ( u ) = ∫ 0 τ I ( T ( η ) , u ) d η ≈ K 1 k = 0 ∑ K − 1 I ( T ( η k ) , u ) CRF Learnable camera response function (CRF) bridges the HDR image and the LDR space by surpress the overexposure or underexposure. C R F l e a k y ( I ~ ( u ) ) = { α I ~ ( u ) , i f I ~ ( u ) 0 I n t e r p ( I ~ ( u ) , Q ) i f 0 ≤ I ~ ( u ) ≤ 1 , − α I ~ ( u ) + α + 1 , i f I ~ ( u ) > 1 \\begin{aligned} \\mathrm{CRF}_\\mathrm{leaky}(\\tilde{\\boldsymbol{I}}(u)) & = \\begin{cases} \\alpha\\tilde{\\boldsymbol{I}}(\\boldsymbol{u}), & \\mathrm{if~}\\tilde{\\boldsymbol{I}}(\\boldsymbol{u}) 1 & \\end{cases} \\end{aligned} CRF leaky ( I ~ ( u )) = ⎩ ⎨ ⎧ α I ~ ( u ) , Interp ( I ~ ( u ) , Q ) − I ~ ( u ) α + α + 1 , if I ~ ( u ) 0 if 0 ≤ I ~ ( u ) ≤ 1 , if I ~ ( u ) > 1 Event Loss Event Frame E k ( u ) E_k(u) E k ( u ) is the ground truth event frame by aggerating the events in the exposure time τ \\tau τ E ^ k ( u ) = log ( B ^ ( T ( η k ) , u ) ) − log ( B ^ ( T ( η k − 1 ) , u ) ) \\hat{E}_k(\\boldsymbol{u})=\\log(\\hat{B}(T(\\eta_k),\\boldsymbol{u}))-\\log(\\hat{B}(T(\\eta_{k-1}),\\boldsymbol{u})) E ^ k ( u ) = lo g ( B ^ ( T ( η k ) , u )) − lo g ( B ^ ( T ( η k − 1 ) , u )) , where B ^ \\hat{B} B ^ is the predicted grayscale image from the Gaussian map. And the rendered event frame is the log difference of the two consecutive frames. Loss Function Here θ \\theta θ is the predefined threshold to trigger events, which is also the parameter to scale the event data and the brightness change. L H E = 1 K ∑ n = 0 K − 1 ∑ E k ( u ) ≠ 0 ∥ θ ⋅ E k ( u ) − E ^ k ( u ) ∥ 1 L_{HE}=\\frac{1}{K}\\sum_{n=0}^{K-1}\\sum_{\\boldsymbol{E}_{k}(\\boldsymbol{u})\\neq0}\\left\\|\\theta\\cdot E_{k}(\\boldsymbol{u})-\\hat{E}_{k}(\\boldsymbol{u})\\right\\|_{1} L H E = K 1 n = 0 ∑ K − 1 E k ( u ) = 0 ∑ θ ⋅ E k ( u ) − E ^ k ( u ) 1 The “No-Event” loss is used to penalize the event prediction when there are no events in the current frame to surpress the artifacts and accerate convergence. L N E = 1 K ∑ n = 0 K − 1 ∑ E k ( u ) = 0 ∥ E ^ k ( u ) ∥ 1 L_{NE}=\\frac{1}{K}\\sum_{n=0}^{K-1}\\sum_{\\boldsymbol{E}_{k}(\\boldsymbol{u})=0}\\left\\|\\hat{E}_{k}(u)\\right\\|_{1} L N E = K 1 n = 0 ∑ K − 1 E k ( u ) = 0 ∑ E ^ k ( u ) 1 Total event loss: L E = L H E + λ N E L N E L_E=L_{HE}+\\lambda_{NE}L_{NE} L E = L H E + λ N E L N E Depth Loss We choose the minimum depth value among the k bins: L D = min k ∥ D o b s − D ( T ( η k ) ) ∥ 1 L_D=\\min_k\\|D_{\\mathrm{obs}}-\\mathcal{D}(T(\\eta_k))\\|_1 L D = k min ∥ D obs − D ( T ( η k )) ∥ 1 Pose Optimization T 0 ∗ , T τ ∗ = arg min T 0 , T τ ( λ E L E + λ I D ( λ I L I + λ D L D ) ) T_{0}^{*},T_{\\tau}^{*}=\\arg\\min_{T_{0},T_{\\tau}}\\left(\\lambda_{E}L_{E}+\\lambda_{ID}\\left(\\lambda_{I}L_{I}+\\lambda_{D}L_{D}\\right)\\right) T 0 ∗ , T τ ∗ = ar g T 0 , T τ min ( λ E L E + λ I D ( λ I L I + λ D L D ) ) Mapping Keyframe Management Calculate IoU between current frame and previous key frame Remove historical key frames having low IoU with current If no frame removed, remove the most redundant frame 3DGS Map Updating After the keyframe is selected, we will update the 3D Gaussian map by: L m a p = λ i s o L i s o + ∑ w ∈ W ′ ( λ E L E w + λ I D ( λ I L I w + λ D L D w ) ) L_{map}=\\lambda_{iso}L_{iso}+\\sum_{w\\in\\mathbf{W}^{\\prime}}(\\lambda_{E}L_{E}^{w}+\\lambda_{ID}(\\lambda_{I}L_{I}^{w}+\\lambda_{D}L_{D}^{w})) L ma p = λ i so L i so + w ∈ W ′ ∑ ( λ E L E w + λ I D ( λ I L I w + λ D L D w )) Need to note that L i s o = 1 ∣ G ∣ ∑ i = 1 ∣ G ∣ ∥ s i − 1 ⋅ s ‾ i ∥ L_{iso}=\\frac{1}{|\\mathbb{G}|}\\sum_{i=1}^{|\\mathbb{G}|}\\|s_i-1\\cdot\\overline{s}_i\\| L i so = ∣ G ∣ 1 ∑ i = 1 ∣ G ∣ ∥ s i − 1 ⋅ s i ∥ , which is to penalize the skinny Gaussians. Experiment Replica This dataset is synthesized, and it is used for evaluating the performance of the proposed method. DEVD This dataset is real-world and created by the authors, which contains 6 scenes with different motion speeds."},{"title":"Efficient Algorithms and Intractable Problems","url":"/posts/51833.html","date":"2025-08-19","updated":"2025-08-19","tags":["Berkeley","Study","Algorithms","CS170"],"categories":["Study","Berkeley","CS"],"excerpt":"Introduction Although the content of this class covers many knowledge that OIers have learned, I still gain some interesting insights from it, like a new way to Vertex Contraction, brief explanation of short path algorithms… Master Theorem ","content":"Introduction Although the content of this class covers many knowledge that OIers have learned, I still gain some interesting insights from it, like a new way to Vertex Contraction, brief explanation of short path algorithms… Master Theorem Master Theorem Graph Vertex Contraction Kosaraju’s Algorithm Reverse the graph. DFS the reversed graph and get the finishing time of each node. DFS the original graph according to the finishing time. The nodes in the same SCC will be visited in the same DFS. Shortest Path 1. Dijkstra’s Algorithm We can imagine the algorithm as water flowing from the source to the destination. The speed of the flowing is 1 distance/s and theorically we will check whether nodes are immersed at each time. 2. Bellman-Ford Algorithm This algorithm mainly solve the problem of negative weight edges. the outest loop is the number of hops, and if after V-1 times, the distance is still updated, then there must be a negative cycle since there must be a circle in this path. Minimum Spanning Tree Easy way to check whether it belongs to MST Yes: Smallest edge between two cuts; No: The biggest edge in the cycle."},{"title":"EvGGS","url":"/posts/20ea9dad.html","date":"2025-08-01","updated":"2025-08-19","tags":["Paper","Computer Vision","3D Reconstruction","3DGS","Gaussian Splatting","Event Camera"],"categories":["Paper","Computer Vision","3D Reconstruction"],"excerpt":"EvGGS Type: Paper Notebook: Paper ( https://www.notion.so/Paper-17de7e7bfd4c80e2bba1f0fe3a6c1131?pvs=21 ) Related Notes: Event Camera ( https://www.notion.so/Event-Camera-1bde7e7bfd4c80549635dcce48628146?pvs=21 ) Paper Trajectory Reconstruc","content":"EvGGS Type: Paper Notebook: Paper ( https://www.notion.so/Paper-17de7e7bfd4c80e2bba1f0fe3a6c1131?pvs=21 ) Related Notes: Event Camera ( https://www.notion.so/Event-Camera-1bde7e7bfd4c80549635dcce48628146?pvs=21 ) Paper Trajectory Reconstruct the 3D Gaussians of scenes in a feedforward manner from the given event stream captured by a moving event camera(to collect multiviews). Input & Output Input the asynchronous event stream and encode it into 3D spatial-temporal voxel grid target view Output 3D Gaussian representation Methodology Monocular Depth Estimation Input the consecutive event voxel grid E k E_k E k and E k − 1 E_{k-1} E k − 1 U-net ϕ d \\phi_d ϕ d will predict disparity map(视差 ): The final depth D p r e d D_{pred} D p r e d can be converted form the disparity Foreground mask (D): filter out the useless backgrounds Intensity Reconstruction Input the event voxel grid, event frame and depth feature volume F d F_d F d Also use UNet ϕ I \\phi_I ϕ I to recover the intensity by multiplying the foreground mask, where F I F_I F I is the output feture volume Gaussian Parameter Regression And the scale, rotation, opacity are calculated Installation conda env create --file environment.yml delete diff-gaussian-rasterization==0.0.0 change to - simple-knn if you wanna update the env with yml after modifying, you can use: conda env update --file environment.yml conda activate evggs ERROR: CUDA_HOME environment variable is not set. Please set it to your CUDA install root. conda install -c conda-forge cudatoolkit-dev -y (can be checked by nvcc —version) git clone [https://github.com/graphdeco-inria/gaussian-splatting](https://github.com/graphdeco-inria/gaussian-splatting) --recursive cd gaussian-splatting/ pip install -e submodules/diff-gaussian-rasterization cd .. Data item = { 'cim' : int_img . astype ( np . float32 ) [ cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] [ np . newaxis ] , #[1, H, W] 'lim' : left_img . astype ( np . float32 ) [ cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] [ np . newaxis ] , 'rim' : right_img . astype ( np . float32 ) [ cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] [ np . newaxis ] , 'leframe' : left_event_frame . transpose ( ( 2 , 0 , 1 ) ) . astype ( np . float32 ) [ : , cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] , #[3, H, W] 'reframe' : right_event_frame . transpose ( ( 2 , 0 , 1 ) ) . astype ( np . float32 ) [ : , cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] , 'ceframe' : center_event_frame . transpose ( ( 2 , 0 , 1 ) ) . astype ( np . float32 ) [ : , cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] , 'lmask' : left_mask [ cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] , #[H, W] 'rmask' : right_mask [ cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] , 'cmask' : center_mask [ cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] , 'lpose' : left_pose . astype ( np . float32 ) , #[4, 4] 'rpose' : right_pose . astype ( np . float32 ) , 'intrinsic' : intrinsic . astype ( np . float32 ) , #[4, 4] 'ldepth' : left_depth_gt . astype ( np . float32 ) [ cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] , # #[H, W] 'rdepth' : right_depth_gt . astype ( np . float32 ) [ cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] , 'cdepth' : center_depth_gt . astype ( np . float32 ) [ cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] , 'center_voxel' : center_event_voxel [ : , cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] , #[5, H, W] 'right_voxel' : right_event_voxel [ : , cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] , 'left_voxel' : left_event_voxel [ : , cs [ 0 ] : cs [ 1 ] , cs [ 2 ] : cs [ 3 ] ] , ### target view rendering parameters ### \"H\" : self . cropped_H , \"W\" : self . cropped_W , \"FovX\" : focal2fov ( intrinsic [ 0 , 0 ] , self . cropped_W ) , \"FovY\" : focal2fov ( intrinsic [ 1 , 1 ] , self . cropped_H ) , 'world_view_transform' : world_view_transform , #[4, 4] 'full_proj_transform' : full_proj_transform , #[4, 4] 'camera_center' : camera_center #[3] }"},{"title":"Hexo Wiki","url":"/posts/12c56117.html","date":"2025-02-14","updated":"2025-08-19","tags":["Wiki","Hexo","Blog"],"categories":["Wiki","Blog"],"excerpt":"Prefix abbrlink : Automatically generated when the post is created. You can jump to another post like this: [Prefix in Hexo Blog](12c56117) to get the link of this post Hexo Wiki . mathjax : If you want to apply equations in your post, plea","content":"Prefix abbrlink : Automatically generated when the post is created. You can jump to another post like this: [Prefix in Hexo Blog](12c56117) to get the link of this post Hexo Wiki . mathjax : If you want to apply equations in your post, please add mathjax: true in the front. cover : The cover image of the post. You can add it by cover: /images/xxx.jpg . toc : Table of Contents. If you want to add a TOC in your post, please add toc: true in the front. top : If you want to make this post stay on the top of the blog, please add top: true in the front. Usage hexo new [post/draft] [title] : Create a new post or draft. The title should be enclosed in quotes if it contains spaces. hexo clean : Clean the cache files (db.json) and generated files (public). hexo generate : Can be simplified to hexo g . Generate static files. hexo server : Can be simplified to hexo s . Start the server (usually at http://localhost:4000/ ). Deploy the blog on Github Page: Configure the info in _config.yml : deploy : type : git repo : [ example : https : //github.com/hexojs/hexojs.github.io ] branch : main token : [ Your Github Token ] message : [ Your commit message ] Type hexo deploy or hexo d in CLI and wait for several minutes. DIY Add Math Formula To use math formulas in your Hexo blog, you can install the hexo-renderer-markdown-it package and configure it accordingly. Make sure to follow the installation instructions in the documentation here . In my own blog, the formula appears twice in different types, so I check f12 and hide the .katex-html class. Hope it can help."},{"title":"Master Theorem","url":"/posts/25951.html","date":"2024-09-03","updated":"2025-08-19","tags":["Study","Algorithms","CS170","Master Theorem"],"categories":["Study","Berkeley","CS"],"excerpt":"Introduction Suppose a recursive function T ( n ) T(n) T ( n ) is defined as follows: T ( n ) = a T ( n / b ) + f ( n ) T(n) = aT(n/b) + f(n) T ( n ) = a T ( n / b ) + f ( n ) , where a ≥ 1 a \\geq 1 a ≥ 1 and b > 1 b > 1 b > 1 are constants","content":"Introduction Suppose a recursive function T ( n ) T(n) T ( n ) is defined as follows: T ( n ) = a T ( n / b ) + f ( n ) T(n) = aT(n/b) + f(n) T ( n ) = a T ( n / b ) + f ( n ) , where a ≥ 1 a \\geq 1 a ≥ 1 and b > 1 b > 1 b > 1 are constants and f ( n ) f(n) f ( n ) is a given function. It means we divide the problem into a a a subproblems, each of size n / b n/b n / b , and the cost of dividing and combining the subproblems is f ( n ) f(n) f ( n ) . Theorem If T ( n ) = a T ( n / b ) + O ( n d ) T(n) = aT(n/b) + O(n^d) T ( n ) = a T ( n / b ) + O ( n d ) , then: $$ T(n)=\\left\\{ \\begin{aligned} &O(n^{log_ba}) & \\text{if } a>b^d \\\\ &O(n^dlog n) & \\text{if } a=b^d \\\\ &O(n^d)& \\text{if } a Preliminary Proof After reconsideration, I found that the initial proof is just kidding. Here is a detailed proof: https://www.luogu.com.cn/article/w3avh1ku Notation O O O : Big O notation, which is used to describe the upper bound of a function. Typically, when x → ∞ x\\rightarrow\\infty x → ∞ , f ( x ) ≤ M ⋅ g ( x ) f(x)\\leq M\\cdot g(x) f ( x ) ≤ M ⋅ g ( x ) (M is a constant). Θ \\Theta Θ : Theta notation, which is used to describe the tight bound of a function. Ω \\Omega Ω : Omega notation, which is used to describe the lower bound of a function. We can draw a recursion tree to analyze the time complexity of the recursive function: Number Time 1 f ( n ) f(n) f ( n ) a a a f ( n / b ) f(n/b) f ( n / b ) a 2 a^2 a 2 f ( n / b 2 ) f(n/b^2) f ( n / b 2 ) ⋯ \\cdots ⋯ ⋯ \\cdots ⋯ a l o g b n a^{log_b n} a l o g b n f ( 1 ) f(1) f ( 1 ) If f ( n ) = n d f(n)=n^d f ( n ) = n d , we get $$\\begin{aligned}T(n) =& f(n) + af(\\frac{n}{b}) + a^2f(\\frac{n}{b^2}) + \\cdots + a^{log_b n}f(1)\\ =&f(n)(1+\\frac{a}{b^d}+\\frac{a^2}{b^{2d}}+\\cdots+\\frac{a^{log_b n}}{b^{dlog_b n}})\\end{aligned}$$ Suppose q = a b d q=\\frac{a}{b^d} q = b d a , then T ( n ) = f ( n ) ( 1 + q + q 2 + ⋯ + q l o g b n ) T(n)=f(n)(1+q+q^2+\\cdots+q^{log_b n}) T ( n ) = f ( n ) ( 1 + q + q 2 + ⋯ + q l o g b n ) . We can use the geometric series formula to simplify the equation: $$T(n)=f(n)\\frac{1-q^{log_b n+1}}{1-q}$$ q > 1 q>1 q > 1 : Noticing that q l o g b n = n l o g b q q^{log_b n}=n^{log_b q} q l o g b n = n l o g b q , so T ( n ) ≈ f ( n ) n l o g b q = f ( n ) n d ⇒ T ( n ) = Θ ( n d ) T(n)\\approx f(n)n^{log_b q}=f(n)n^d\\Rightarrow T(n)=\\Theta(n^d) T ( n ) ≈ f ( n ) n l o g b q = f ( n ) n d ⇒ T ( n ) = Θ ( n d ) . q = 1 q=1 q = 1 : T ( n ) = f ( n ) l o g b n ⇒ T ( n ) = Θ ( n d l o g n ) T(n)=f(n)log_b n\\Rightarrow T(n)=\\Theta(n^dlog n) T ( n ) = f ( n ) l o g b n ⇒ T ( n ) = Θ ( n d l o g n ) . q 1 q q 1 : T ( n ) = f ( n ) 1 − q l o g b n + 1 1 − q ≤ f ( n ) 1 − q ⇒ T ( n ) = Θ ( f ( n ) ) T(n)=f(n)\\frac{1-q^{log_b n+1}}{1-q}\\leq\\frac{f(n)}{1-q}\\Rightarrow T(n)=\\Theta(f(n)) T ( n ) = f ( n ) 1 − q 1 − q l o g b n + 1 ≤ 1 − q f ( n ) ⇒ T ( n ) = Θ ( f ( n )) . Application Large number addition: T ( n ) = 3 T ( n / 2 ) + O ( n ) T(n)=3T(n/2)+O(n) T ( n ) = 3 T ( n /2 ) + O ( n ) (We can divide each number into two parts and use three registers to calculate the following additions), a = 3 a=3 a = 3 , b = 2 b=2 b = 2 , f ( n ) = O ( n ) f(n)=O(n) f ( n ) = O ( n ) , a > b d a>b^d a > b d , T ( n ) = Θ ( n l o g 2 3 ) T(n)=\\Theta(n^{log_2 3}) T ( n ) = Θ ( n l o g 2 3 ) . Merge sort: T ( n ) = 2 T ( n / 2 ) + O ( n ) T(n)=2T(n/2)+O(n) T ( n ) = 2 T ( n /2 ) + O ( n ) , a = 2 a=2 a = 2 , b = 2 b=2 b = 2 , f ( n ) = O ( n ) f(n)=O(n) f ( n ) = O ( n ) , a = b d a=b^d a = b d , T ( n ) = Θ ( n log n ) T(n)=\\Theta(n\\log n) T ( n ) = Θ ( n lo g n ) ."},{"title":"Silicon Valley","url":"/posts/5045.html","date":"2024-09-01","updated":"2025-08-19","tags":["Travel","Silicon Valley","Stanford","Apple Park"],"categories":["Travel","California"],"excerpt":"Thanks to rockyyh , who inspired me to set up this blog website. And the blog is decided to be written in English. Since this labor day is at 2nd September, we decided to visit the Silicon Valley. Stanford University Campus Departing late m","content":"Thanks to rockyyh , who inspired me to set up this blog website. And the blog is decided to be written in English. Since this labor day is at 2nd September, we decided to visit the Silicon Valley. Stanford University Campus Departing late morning, we chose Lyft as our transportation since the public transportation costs us 12.5$ each, which is almost the same. All of us were struck by the large scale of Stanford, having a long road with tall palms on both sides, a grassland covering the whole eyes, and a large court before the “useful” buildings. The whole campus is scattered with buildings, and I believe the major part is the greenery. The buildings are not tall, but they are all in the same style, a bit like the map of Anubis in CS:GO. Immersed in this atmosphere, I believe the environment will affect the mental state of a man. Cantor Arts Center After having a wonderful self-service meal, which is a bit expensive, we visited the art museum nearby. At first, I thought the museum is just composed of exhibitions of each continent, although the china or even the bronze implement truly impressed me. But there are also many modern artworks: A “crude” model of a man in African lobby was made in 19th century. There are also small rooms to record some artists not that famous, and a woman of Chinese American origin was one of them. She also left some drafts like demonstrating the Chinese characters. Wandering around the museum, I began to notice some familiar sculptures. And a sculpture of an ugly but thoughtful man appeared in front of me. And I finally realized that it’s Balzac made by Rodin. Even after getting out of the campus, I remembered that the sculpture group in Memorial Court is the Burghers of Calais! Hoover Tower The 8$ ticket is a bit expensive, and the above landscape didn’t amaze me, maybe the color is too uniformed. But the staff talked with us a lot. Once the woman (controling the elevator) saw us wearing CAL clothes and hat, she began mocking: “You guy cals…” And another civilised elder executive introduced the stories about the bell set in the tower. Once her friend didn’t leave before the bell performance, and she was scared to death. To our surprise, the bell is still working and there was a performance just at 6:00pm. And above the elevator button, a analog dial can precisely display the elevator’s current layer after almost a hundred year. Apple Park Borrrrrring and dulllllll! Without invitation, we can only wandering in the visitor center, having a drink, glancing at the productions that I can afford, and playing the iPad with weak AR of the sandbox. However, we wasted too much in Stanford so that the Google headquarter and the Computer History Museum couldn’t be our next destination. Hope next time I can visit not just as a visitor."},{"title":"VGGT","url":"/posts/69c95808.html","date":"2025-08-02","updated":"2025-08-19","tags":["Paper","Computer Vision","3D Reconstruction","Vision Foundation Model"],"categories":["Paper","Computer Vision","3D Reconstruction"],"excerpt":"Introduction An end-to-end transformer model, conceptually similar to Stable Diffusion or GPT but applied to a different task. It takes a sequence of images as input. It outputs camera poses, depth maps, point maps, and tracking information","content":"Introduction An end-to-end transformer model, conceptually similar to Stable Diffusion or GPT but applied to a different task. It takes a sequence of images as input. It outputs camera poses, depth maps, point maps, and tracking information. It can process hundreds of images within one second. It infers the complete set of 3D attributes without requiring post-optimization. Background Traditional 3D reconstruction utilize visual-geometry methods (BA) BA (Bundle Adjustment) is a mathematical optimization technique to refine the 3D structure and camera positions by minimizing reprojection errors across multiple views. 最小化重投影误差 LM算法使得观测的图像点坐标与预测的图像点坐标之间的误差最小 Machine learning method: address tasks like feature matching and monocular depth prediction VGGsfm: integrate the deep learning model into the SfM(structure from motion) to achieve end-to-end VGGT: predicts a full set of 3D attributes, including camera parameters, depth maps, point maps, and 3D point tracks. Tracking Any Point: track the POI in the video sequence Model Architecture Aggregator The model first processes input images using an aggregator. DINO: Used for image patchification to get local tokens ( t I t_I t I ). Augmentation tokens: A camera token ( t g t_g t g ) is added. A register token ( t R t_R t R ) is added. Concat: The tokens are concatenated, and positional embeddings are added. Alternating Attention The token sequence is processed through L layers of alternating attention blocks. Global Attention: Attention is computed across tokens from all frames. Tensor shape: [B, N*P, C]. Frame Attention: Attention is computed among tokens within each individual frame. Tensor shape: [B*N, P, C]. The initial camera token ( t _ 1 g t\\_{1g} t _ 1 g ) and register token (t_1R) are learnable, allowing the model to set the first frame as the coordinate system. The register token (t_R) is discarded during the prediction phase. Prediction Heads After the attention blocks, the tokens are passed to different prediction heads to generate the final outputs. Camera Head: Uses the camera token (t_ig). Predicts the camera parameters (g) using self-attention and a linear layer. DPT Head: Uses the image tokens (t_iI). Employs a DPT (Dense Prediction Transformer) to get feature maps F_i (with shape CxHxW). A 3x3 convolution predicts the depth map (D_i), point map (P_i), tracking features (T_i), and a confidence map. Track Head: For a given query point (y_j) in a query image (I_q). It performs a bilinear sample on the feature map (T_q) to get the feature (f_y). This feature is correlated with other feature maps (T_i) to get correlation maps. These maps are fed into a self-attention layer to predict the corresponding 2D points (y_i). Training The model is trained with a composite loss function. The camera loss uses the Huber Loss function. Total Loss: L = L c a m e r a + L d e p t h + L p m a p + λ L t r a c k L=L_{camera}+L_{depth}+L_{pmap}+\\lambda L_{track} L = L c am er a + L d e pt h + L p ma p + λ L t r a c k Camera Loss: L c a m e r a = ∑ i = 1 N ∥ g ^ i − g i ∥ c L_{camera} = \\sum_{i=1}^{N} \\left\\| \\hat{g}_{i} - g_{i} \\right\\|_{c} L c am er a = i = 1 ∑ N ∥ g ^ i − g i ∥ c Depth Loss: L d e p t h = ∑ i = 1 N ∣ ∑ i p ⊙ ( D ^ i − D i ) ∣ + ∣ ∑ i p ⊙ ( ∇ D ^ i − ∇ D i ) ∣ − α log ∑ i p L_{depth} = \\sum_{i=1}^{N} \\left| \\sum_{ip} \\odot (\\hat{D}_{i} - D_{i}) \\right| + \\left| \\sum_{ip} \\odot (\\nabla \\hat{D}_{i} - \\nabla D_{i}) \\right| - \\alpha \\log \\sum_{ip} L d e pt h = i = 1 ∑ N i p ∑ ⊙ ( D ^ i − D i ) + i p ∑ ⊙ ( ∇ D ^ i − ∇ D i ) − α lo g i p ∑ Point Map Loss: L p m a p = ∑ i = 1 N ∣ ∑ i p ⊙ ( P ^ i − P i ) ∣ + ∣ ∑ i p ⊙ ( ∇ P ^ i − ∇ P i ) ∣ − α log ∑ i p L_{pmap} = \\sum_{i=1}^{N} \\left| \\sum_{ip} \\odot (\\hat{P}_{i} - P_{i}) \\right| + \\left| \\sum_{ip} \\odot (\\nabla \\hat{P}_{i} - \\nabla P_{i}) \\right| - \\alpha \\log \\sum_{ip} L p ma p = i = 1 ∑ N i p ∑ ⊙ ( P ^ i − P i ) + i p ∑ ⊙ ( ∇ P ^ i − ∇ P i ) − α lo g i p ∑ Track Loss: L t r a c k = ∑ j = 1 M ∑ i = 1 N ∣ y j , i − y ^ j , i ∣ L_{track} = \\sum_{j=1}^{M} \\sum_{i=1}^{N} \\left| y_{j,i} - \\hat{y}_{j,i} \\right| L t r a c k = j = 1 ∑ M i = 1 ∑ N ∣ y j , i − y ^ j , i ∣ Limitations Does not support fisheye or panoramic images. Reconstruction quality is low when input images have extreme rotations. Inference memory usage increases rapidly as the number of input images grows."},{"title":"Vim","url":"/posts/44303.html","date":"2024-09-22","updated":"2025-08-19","tags":["Linux","Command","Vim","Editor"],"categories":["Tools","Editor"],"excerpt":"Mode Switching i/a/o : Enter insert mode (insert before, after, or on a new line). Esc : Exit insert mode and return to normal mode. v : Enter visual mode (highlight text). V : Enter visual line mode (highlight entire lines). Ctrl + v : Ent","content":"Mode Switching i/a/o : Enter insert mode (insert before, after, or on a new line). Esc : Exit insert mode and return to normal mode. v : Enter visual mode (highlight text). V : Enter visual line mode (highlight entire lines). Ctrl + v : Enter visual block mode (highlight blocks of text). File Operations :w : Save the file. :wq : Save and quit. :q : Quit the editor. :q! : Force quit without saving. :set number : Show line numbers. :set nonumber : Hide line numbers. :e filename : Open a file in Vim. :r filename : Insert the content of another file into the current file. :! command : Execute an external command (e.g., :!ls to list files). Cursor Movement h/j/k/l : Move the cursor left, down, up, and right. b : Move to the beginning of the current/previous word. e : Move to the end of the current/next word. w : Move to the beginning of the next word. 0 : Move to the beginning of the line. $: Move to the end of the line. G : Move to the end of the file. gg : Move to the beginning of the file. Ctrl + f : Scroll forward one page. Ctrl + b : Scroll backward one page. H : Move to the top of the screen. M : Move to the middle of the screen. L : Move to the bottom of the screen. Ctrl + o : Jump back to the previous cursor position. Ctrl + i : Jump forward to the next cursor position. Editing Operations dd : Delete a line. d motion: Delete text following a movement (e.g., dw to delete a word). x : Delete a character. u : Undo the last operation. Ctrl + r : Redo the last undone operation. yy : Copy (yank) a line. y motion: Copy text following a movement (e.g., yw to copy a word). p : Paste the copied or deleted text. r : Replace a single character under the cursor. cw : Change (delete and enter insert mode) a word. ci : Change inside quotes. >> : Indent a line. Un-indent a line. Search and Replace /pattern : Search for a pattern. ?pattern : Search for a pattern in reverse. n : Repeat the search in the same direction. N : Repeat the search in the opposite direction. :%s/old/new/g : Replace all occurrences of “old” with “new” in the file. :noh : Clear search highlighting. Buffer and Window Management :bn : Go to the next buffer. :bp : Go to the previous buffer. :bd : Close the current buffer. :split : Split the window horizontally. :vsplit : Split the window vertically. Ctrl + w w : Switch between windows."}]