https://blog.cptz.space/posts/build-your-own-image-bed/ 2023-05-10T20:55:00+08:00 https://blog.cptz.space/posts/fundamentals-of-distributed-systems/ 2023-05-12T20:23:15+08:00 https://blog.cptz.space/posts/nccl-learning-series/ 2026-07-09T09:00:00+08:00 https://blog.cptz.space/posts/nccl-stack-topology/ 2026-07-09T09:05:00+08:00 https://blog.cptz.space/posts/nccl-rank-process-group/ 2026-07-09T09:10:00+08:00 https://blog.cptz.space/posts/nccl-stack-version-boundary/ 2026-07-10T09:00:00+08:00 https://blog.cptz.space/posts/nccl-cuda-async-completion/ 2026-07-10T09:10:00+08:00 https://blog.cptz.space/posts/nccl-collectives/ 2026-07-10T09:20:00+08:00 https://blog.cptz.space/posts/nccl-group-ordering/ 2026-07-10T09:30:00+08:00 https://blog.cptz.space/posts/nccl-collective-decomposition/ 2026-07-10T09:40:00+08:00 https://blog.cptz.space/posts/nccl-performance-model-statistics/ 2026-07-10T09:50:00+08:00 https://blog.cptz.space/posts/nccl-tests-benchmark/ 2026-07-10T10:00:00+08:00 https://blog.cptz.space/posts/nccl-latency-bandwidth-model/ 2026-07-10T10:10:00+08:00 https://blog.cptz.space/posts/nccl-parameter-sweeps/ 2026-07-10T10:20:00+08:00 https://blog.cptz.space/posts/nccl-ring-allreduce-steps/ 2026-07-10T10:30:00+08:00 https://blog.cptz.space/posts/nccl-tree-double-tree-hierarchy/ 2026-07-10T16:20:00+08:00 https://blog.cptz.space/posts/nccl-simple-ll-ll128-protocols/ 2026-07-10T17:10:00+08:00 https://blog.cptz.space/posts/nccl-channel-chunk-slice-step-buffer/ 2026-07-10T20:00:00+08:00 https://blog.cptz.space/posts/nccl-tuning-model-auto-selection-plugin/ 2026-07-10T20:30:00+08:00 https://blog.cptz.space/posts/nccl-bootstrap-communicator-init-state-machine/ 2026-07-11T10:00:00+08:00 https://blog.cptz.space/posts/nccl-topology-xml-path-computation/ 2026-07-11T12:00:00+08:00 https://blog.cptz.space/posts/nccl-graph-search-ring-tree-channel-construction/ 2026-07-11T15:00:00+08:00 https://blog.cptz.space/posts/nccl-transport-selection-connector-lifecycle/ 2026-07-11T20:00:00+08:00 https://blog.cptz.space/posts/nccl-p2p-cuda-ipc-shm-copy-engine/ 2026-07-11T21:30:00+08:00 https://blog.cptz.space/posts/nccl-enqueue-task-plan-work-batch-cuda-graph/ 2026-07-11T22:30:00+08:00 https://blog.cptz.space/posts/nccl-device-kernel-primitives-step-flag/ 2026-07-11T22:50:00+08:00 https://blog.cptz.space/posts/nccl-cpu-proxy-net-progress-socket/ 2026-07-11T23:50:00+08:00 https://blog.cptz.space/posts/nccl-socket-infiniband-roce-verbs/ 2026-07-11T23:59:00+08:00 https://blog.cptz.space/posts/nccl-gpudirect-rdma-mr-registration-cache/ 2026-07-12T00:10:00+08:00 https://blog.cptz.space/posts/nccl-multinic-rail-crossnic-pxn/ 2026-07-12T00:25:00+08:00 https://blog.cptz.space/posts/nccl-collnet-sharp-nvls-plugin-fallback/ 2026-07-12T00:35:00+08:00 https://blog.cptz.space/posts/nccl-processgroupnccl-stream-event-work/ 2026-07-12T00:50:00+08:00 https://blog.cptz.space/posts/nccl-ddp-reducer-bucket-overlap/ 2026-07-12T01:10:00+08:00 https://blog.cptz.space/posts/nccl-zero-fsdp-state-sharding/ 2026-07-12T01:30:00+08:00 https://blog.cptz.space/posts/nccl-tp-pp-ep-multidim-process-groups/ 2026-07-12T02:00:00+08:00 https://blog.cptz.space/posts/nccl-collective-mismatch-hang-rank-crash-root-cause/ 2026-07-12T02:20:00+08:00 https://blog.cptz.space/posts/nccl-debug-nvtx-nsight-flight-recorder/ 2026-07-12T02:30:00+08:00 https://blog.cptz.space/posts/nccl-production-baseline-regression-version-governance/ 2026-07-12T02:35:00+08:00 https://blog.cptz.space/posts/nccl-allreduce-end-to-end-capstone/ 2026-07-12T02:40:00+08:00 https://blog.cptz.space/posts/nccl-interview-guide/ 2026-07-15T09:00:00+08:00 https://blog.cptz.space/posts/nccl-interview-deep-dive-labs/ 2026-07-17T16:30:00+08:00 https://blog.cptz.space/posts/vllm-sglang-interview-series/ 2026-07-20T00:10:00+08:00 https://blog.cptz.space/posts/vllm-sglang-interview-level-1-foundations/ 2026-07-20T00:11:00+08:00 https://blog.cptz.space/posts/vllm-sglang-interview-level-2-lifecycle/ 2026-07-20T00:12:00+08:00 https://blog.cptz.space/posts/vllm-sglang-interview-level-3-scheduling/ 2026-07-20T00:13:00+08:00 https://blog.cptz.space/posts/vllm-sglang-interview-level-4-kv-runner/ 2026-07-20T00:14:00+08:00 https://blog.cptz.space/posts/vllm-sglang-interview-level-5-advanced-distributed/ 2026-07-20T00:15:00+08:00 https://blog.cptz.space/posts/vllm-sglang-interview-level-6-system-design/ 2026-07-20T00:16:00+08:00 https://blog.cptz.space/posts/vllm-sglang-interview-comprehensive-guide/ 2026-07-20T00:17:00+08:00 https://blog.cptz.space/posts/vllm-sglang-interview-source-labs/ 2026-07-20T00:18:00+08:00 https://blog.cptz.space/posts/vllm-request-lifecycle-source-guide/ 2026-07-20T00:19:00+08:00 https://blog.cptz.space/posts/sglang-request-lifecycle-source-guide/ 2026-07-20T00:20:00+08:00 https://blog.cptz.space/posts/aiinfra-interview-series/ 2026-07-20T16:00:00+08:00 https://blog.cptz.space/posts/aiinfra-interview-level-1-gpu-performance/ 2026-07-20T16:01:00+08:00 https://blog.cptz.space/posts/aiinfra-interview-level-2-training-runtime/ 2026-07-20T16:02:00+08:00 https://blog.cptz.space/posts/aiinfra-interview-level-3-model-parallel/ 2026-07-20T16:03:00+08:00 https://blog.cptz.space/posts/aiinfra-interview-level-4-inference-serving/ 2026-07-20T16:04:00+08:00 https://blog.cptz.space/posts/aiinfra-interview-level-5-kernel-compiler/ 2026-07-20T16:05:00+08:00 https://blog.cptz.space/posts/aiinfra-interview-level-6-production-reliability/ 2026-07-20T16:06:00+08:00 https://blog.cptz.space/posts/aiinfra-interview-level-7-project-defense/ 2026-07-20T16:07:00+08:00 https://blog.cptz.space/posts/aiinfra-interview-level-8-system-design/ 2026-07-20T16:08:00+08:00 https://blog.cptz.space/posts/aiinfra-interview-level-9-engineering-deep-dive/ 2026-07-20T16:09:00+08:00 https://blog.cptz.space/posts/aiinfra-interview-level-10-runtime-contracts/ 2026-07-20T16:10:00+08:00 https://blog.cptz.space/posts/aiinfra-interview-practical-guide/ 2026-07-20T16:11:00+08:00 https://blog.cptz.space/posts/megatron-interview-series/ 2026-07-20T16:12:00+08:00 https://blog.cptz.space/posts/megatron-interview-level-0-architecture/ 2026-07-20T16:13:00+08:00 https://blog.cptz.space/posts/megatron-interview-level-1-model-dataflow/ 2026-07-20T16:14:00+08:00 https://blog.cptz.space/posts/megatron-interview-level-2-tensor-sequence-parallel/ 2026-07-20T16:15:00+08:00 https://blog.cptz.space/posts/megatron-interview-level-3-pipeline-parallel/ 2026-07-20T16:16:00+08:00 https://blog.cptz.space/posts/megatron-interview-level-4-distributed-optimizer/ 2026-07-20T16:17:00+08:00 https://blog.cptz.space/posts/megatron-interview-level-5-context-long-sequence/ 2026-07-20T16:18:00+08:00 https://blog.cptz.space/posts/megatron-interview-level-6-moe-expert-parallel/ 2026-07-20T16:19:00+08:00 https://blog.cptz.space/posts/megatron-interview-level-7-performance-precision/ 2026-07-20T16:20:00+08:00 https://blog.cptz.space/posts/megatron-interview-level-8-checkpoint-reliability/ 2026-07-20T16:21:00+08:00 https://blog.cptz.space/posts/megatron-interview-level-9-expert-systems/ 2026-07-20T16:22:00+08:00 https://blog.cptz.space/posts/megatron-interview-study-labs/ 2026-07-20T16:23:00+08:00 https://blog.cptz.space/posts/vllm-sglang-scheduler-source-deep-dive/ 2026-07-21T00:05:00+08:00 https://blog.cptz.space/posts/nvidia-gpu-volta-to-blackwell/ 2026-07-21T10:30:00+08:00 https://blog.cptz.space/posts/cuda-features-01-unified-memory/ 2026-07-22T08:01:00+08:00 https://blog.cptz.space/posts/cuda-features-02-cuda-graphs/ 2026-07-22T08:02:00+08:00 https://blog.cptz.space/posts/cuda-features-03-stream-ordered-memory-allocator/ 2026-07-22T08:03:00+08:00 https://blog.cptz.space/posts/cuda-features-04-cooperative-groups/ 2026-07-22T08:04:00+08:00 https://blog.cptz.space/posts/cuda-features-05-programmatic-dependent-launch/ 2026-07-22T08:05:00+08:00 https://blog.cptz.space/posts/cuda-features-06-green-contexts/ 2026-07-22T08:06:00+08:00 https://blog.cptz.space/posts/cuda-features-07-lazy-loading/ 2026-07-22T08:07:00+08:00 https://blog.cptz.space/posts/cuda-features-08-error-log-management/ 2026-07-22T08:08:00+08:00 https://blog.cptz.space/posts/cuda-features-09-asynchronous-barriers/ 2026-07-22T08:09:00+08:00 https://blog.cptz.space/posts/cuda-features-10-pipelines/ 2026-07-22T08:10:00+08:00 https://blog.cptz.space/posts/cuda-features-11-asynchronous-data-copies/ 2026-07-22T08:11:00+08:00 https://blog.cptz.space/posts/cuda-features-12-cluster-launch-control/ 2026-07-22T08:12:00+08:00 https://blog.cptz.space/posts/cuda-features-13-l2-cache-control/ 2026-07-22T08:13:00+08:00 https://blog.cptz.space/posts/cuda-features-14-memory-synchronization-domains/ 2026-07-22T08:14:00+08:00 https://blog.cptz.space/posts/cuda-features-15-interprocess-communication/ 2026-07-22T08:15:00+08:00 https://blog.cptz.space/posts/cuda-features-16-virtual-memory-management/ 2026-07-22T08:16:00+08:00 https://blog.cptz.space/posts/cuda-features-17-extended-gpu-memory/ 2026-07-22T08:17:00+08:00 https://blog.cptz.space/posts/cuda-features-18-dynamic-parallelism/ 2026-07-22T08:18:00+08:00 https://blog.cptz.space/posts/cuda-features-19-graphics-interop/ 2026-07-22T08:19:00+08:00 https://blog.cptz.space/posts/cuda-features-20-driver-entry-point-access/ 2026-07-22T08:20:00+08:00 https://blog.cptz.space/categories/ 2026-07-22T11:25:11+08:00 https://blog.cptz.space/tags/ 2026-07-22T11:25:11+08:00 https://blog.cptz.space/archives/ 2026-07-22T11:25:11+08:00 https://blog.cptz.space/about/ 2026-07-22T11:25:11+08:00 https://blog.cptz.space/readme/2019-08-08-text-and-typography.html https://blog.cptz.space/readme/2019-08-08-write-a-new-post.html https://blog.cptz.space/readme/2019-08-09-getting-started.html https://blog.cptz.space/readme/2019-08-11-customize-the-favicon.html https://blog.cptz.space/readme/2021-01-03-enable-google-pv.html https://blog.cptz.space/ https://blog.cptz.space/tags/minio/ https://blog.cptz.space/tags/image-bed/ https://blog.cptz.space/tags/typora/ https://blog.cptz.space/tags/distributed-system/ https://blog.cptz.space/tags/notes/ https://blog.cptz.space/tags/nccl/ https://blog.cptz.space/tags/cuda/ https://blog.cptz.space/tags/gpu/ https://blog.cptz.space/tags/ddp/ https://blog.cptz.space/tags/infra/ https://blog.cptz.space/tags/nvlink/ https://blog.cptz.space/tags/pcie/ https://blog.cptz.space/tags/numa/ https://blog.cptz.space/tags/topology/ https://blog.cptz.space/tags/rank/ https://blog.cptz.space/tags/communicator/ https://blog.cptz.space/tags/process-group/ https://blog.cptz.space/tags/torchrun/ https://blog.cptz.space/tags/cuda-visible-devices/ https://blog.cptz.space/tags/pytorch/ https://blog.cptz.space/tags/elf/ https://blog.cptz.space/tags/dynamic-linker/ https://blog.cptz.space/tags/stream/ https://blog.cptz.space/tags/event/ https://blog.cptz.space/tags/asynchronous/ https://blog.cptz.space/tags/collective/ https://blog.cptz.space/tags/all-reduce/ https://blog.cptz.space/tags/all-gather/ https://blog.cptz.space/tags/reduce-scatter/ https://blog.cptz.space/tags/all-to-all/ https://blog.cptz.space/tags/group/ https://blog.cptz.space/tags/ordering/ https://blog.cptz.space/tags/timeout/ https://blog.cptz.space/tags/watchdog/ https://blog.cptz.space/tags/processgroup/ https://blog.cptz.space/tags/fsdp/ https://blog.cptz.space/tags/zero/ https://blog.cptz.space/tags/nccl-tests/ https://blog.cptz.space/tags/algbw/ https://blog.cptz.space/tags/busbw/ https://blog.cptz.space/tags/benchmark/ https://blog.cptz.space/tags/statistics/ https://blog.cptz.space/tags/performance/ https://blog.cptz.space/tags/latency/ https://blog.cptz.space/tags/bandwidth/ https://blog.cptz.space/tags/tuning/ https://blog.cptz.space/tags/performance-model/ https://blog.cptz.space/tags/regression/ https://blog.cptz.space/tags/transport/ https://blog.cptz.space/tags/channel/ https://blog.cptz.space/tags/troubleshooting/ https://blog.cptz.space/tags/ring/ https://blog.cptz.space/tags/tree/ https://blog.cptz.space/tags/double-tree/ https://blog.cptz.space/tags/allreduce/ https://blog.cptz.space/tags/simple/ https://blog.cptz.space/tags/ll/ https://blog.cptz.space/tags/ll128/ https://blog.cptz.space/tags/protocol/ https://blog.cptz.space/tags/chunk/ https://blog.cptz.space/tags/slice/ https://blog.cptz.space/tags/buffer/ https://blog.cptz.space/tags/algorithm/ https://blog.cptz.space/tags/plugin/ https://blog.cptz.space/tags/bootstrap/ https://blog.cptz.space/tags/debugging/ https://blog.cptz.space/tags/xml/ https://blog.cptz.space/tags/graph/ https://blog.cptz.space/tags/p2p/ https://blog.cptz.space/tags/shm/ https://blog.cptz.space/tags/network/ https://blog.cptz.space/tags/connector/ https://blog.cptz.space/tags/cuda-ipc/ https://blog.cptz.space/tags/cumem/ https://blog.cptz.space/tags/enqueue/ https://blog.cptz.space/tags/planner/ https://blog.cptz.space/tags/cuda-graph/ https://blog.cptz.space/tags/nvtx/ https://blog.cptz.space/tags/nsys/ https://blog.cptz.space/tags/cuda-kernel/ https://blog.cptz.space/tags/primitives/ https://blog.cptz.space/tags/synchronization/ https://blog.cptz.space/tags/proxy/ https://blog.cptz.space/tags/socket/ https://blog.cptz.space/tags/cpu-affinity/ https://blog.cptz.space/tags/nsight/ https://blog.cptz.space/tags/infiniband/ https://blog.cptz.space/tags/roce/ https://blog.cptz.space/tags/rdma/ https://blog.cptz.space/tags/verbs/ https://blog.cptz.space/tags/gpudirect-rdma/ https://blog.cptz.space/tags/dmabuf/ https://blog.cptz.space/tags/memory-registration/ https://blog.cptz.space/tags/iommu/ https://blog.cptz.space/tags/multi-nic/ https://blog.cptz.space/tags/rail/ https://blog.cptz.space/tags/pxn/ https://blog.cptz.space/tags/collnet/ https://blog.cptz.space/tags/sharp/ https://blog.cptz.space/tags/nvls/ https://blog.cptz.space/tags/nvswitch/ https://blog.cptz.space/tags/processgroupnccl/ https://blog.cptz.space/tags/cuda-stream/ https://blog.cptz.space/tags/worknccl/ https://blog.cptz.space/tags/reducer/ https://blog.cptz.space/tags/bucket/ https://blog.cptz.space/tags/overlap/ https://blog.cptz.space/tags/optimizer/ https://blog.cptz.space/tags/sharding/ https://blog.cptz.space/tags/tensor-parallel/ https://blog.cptz.space/tags/pipeline-parallel/ https://blog.cptz.space/tags/expert-parallel/ https://blog.cptz.space/tags/moe/ https://blog.cptz.space/tags/failure/ https://blog.cptz.space/tags/root-cause/ https://blog.cptz.space/tags/debug/ https://blog.cptz.space/tags/nsight-systems/ https://blog.cptz.space/tags/flight-recorder/ https://blog.cptz.space/tags/observability/ https://blog.cptz.space/tags/production/ https://blog.cptz.space/tags/canary/ https://blog.cptz.space/tags/baseline/ https://blog.cptz.space/tags/governance/ https://blog.cptz.space/tags/capstone/ https://blog.cptz.space/tags/callgraph/ https://blog.cptz.space/tags/diagnosis/ https://blog.cptz.space/tags/interview/ https://blog.cptz.space/tags/vllm/ https://blog.cptz.space/tags/sglang/ https://blog.cptz.space/tags/experiment/ https://blog.cptz.space/tags/llm-inference/ https://blog.cptz.space/tags/kv-cache/ https://blog.cptz.space/tags/scheduler/ https://blog.cptz.space/tags/architecture/ https://blog.cptz.space/tags/ai-infra/ https://blog.cptz.space/tags/distributed-training/ https://blog.cptz.space/tags/megatron/ https://blog.cptz.space/tags/megatron-core/ https://blog.cptz.space/tags/model-parallel/ https://blog.cptz.space/tags/continuous-batching/ https://blog.cptz.space/tags/chunked-prefill/ https://blog.cptz.space/tags/nvidia/ https://blog.cptz.space/tags/gpu-architecture/ https://blog.cptz.space/tags/tensor-core/ https://blog.cptz.space/tags/volta/ https://blog.cptz.space/tags/turing/ https://blog.cptz.space/tags/ampere/ https://blog.cptz.space/tags/ada/ https://blog.cptz.space/tags/hopper/ https://blog.cptz.space/tags/blackwell/ https://blog.cptz.space/tags/tma/ https://blog.cptz.space/tags/wgmma/ https://blog.cptz.space/tags/tmem/ https://blog.cptz.space/tags/unified-memory/ https://blog.cptz.space/tags/hmm/ https://blog.cptz.space/tags/memory/ https://blog.cptz.space/tags/cuda-graphs/ https://blog.cptz.space/tags/launch-overhead/ https://blog.cptz.space/tags/runtime/ https://blog.cptz.space/tags/memory-pool/ https://blog.cptz.space/tags/cuda-malloc-async/ https://blog.cptz.space/tags/allocator/ https://blog.cptz.space/tags/cooperative-groups/ https://blog.cptz.space/tags/collectives/ https://blog.cptz.space/tags/dependent-launch/ https://blog.cptz.space/tags/green-context/ https://blog.cptz.space/tags/qos/ https://blog.cptz.space/tags/resource-partition/ https://blog.cptz.space/tags/lazy-loading/ https://blog.cptz.space/tags/module/ https://blog.cptz.space/tags/startup/ https://blog.cptz.space/tags/logging/ https://blog.cptz.space/tags/diagnostics/ https://blog.cptz.space/tags/driver-api/ https://blog.cptz.space/tags/barrier/ https://blog.cptz.space/tags/pipeline/ https://blog.cptz.space/tags/double-buffering/ https://blog.cptz.space/tags/async-copy/ https://blog.cptz.space/tags/cp-async/ https://blog.cptz.space/tags/stas/ https://blog.cptz.space/tags/cluster-launch-control/ https://blog.cptz.space/tags/work-stealing/ https://blog.cptz.space/tags/scheduling/ https://blog.cptz.space/tags/l2-cache/ https://blog.cptz.space/tags/persisting-cache/ https://blog.cptz.space/tags/memory-model/ https://blog.cptz.space/tags/fence/ https://blog.cptz.space/tags/ipc/ https://blog.cptz.space/tags/vmm/ https://blog.cptz.space/tags/multi-gpu/ https://blog.cptz.space/tags/virtual-memory/ https://blog.cptz.space/tags/multicast/ https://blog.cptz.space/tags/egm/ https://blog.cptz.space/tags/nvlink-c2c/ https://blog.cptz.space/tags/grace-hopper/ https://blog.cptz.space/tags/dynamic-parallelism/ https://blog.cptz.space/tags/cdp2/ https://blog.cptz.space/tags/device-runtime/ https://blog.cptz.space/tags/vulkan/ https://blog.cptz.space/tags/opengl/ https://blog.cptz.space/tags/direct3d/ https://blog.cptz.space/tags/nvsci/ https://blog.cptz.space/tags/interop/ https://blog.cptz.space/tags/abi/ https://blog.cptz.space/tags/dynamic-loading/ https://blog.cptz.space/categories/experience/ https://blog.cptz.space/categories/tutorial/ https://blog.cptz.space/categories/reading-notes/ https://blog.cptz.space/categories/%E6%B7%B1%E5%85%A5%E7%90%86%E8%A7%A3%E5%88%86%E5%B8%83%E5%BC%8F%E7%B3%BB%E7%BB%9F/ https://blog.cptz.space/categories/ai-infra/ https://blog.cptz.space/categories/nccl/ https://blog.cptz.space/categories/inference/ https://blog.cptz.space/categories/interview/ https://blog.cptz.space/categories/megatron/ https://blog.cptz.space/categories/gpu/ https://blog.cptz.space/categories/cuda-features/ https://blog.cptz.space/page2/ https://blog.cptz.space/page3/ https://blog.cptz.space/page4/ https://blog.cptz.space/page5/