Menu

#TransFormEr

39 posts

Feed·
20 of 39 posts
Tech & AI Synthesis: Attention and Transformer Architechture — Part 2
📰
0Read More

Tech & AI Synthesis: Attention and Transformer Architechture — Part 2

DEV Community: deeplearning·Sharath·2 days ago
#YyZqFXSH

In this part of blog we will emphasise our focus on Transformer Architechture more and understand what is encoder, decoder… Continue reading on Medium »

15s
0Read More

I merged attention and MLP processing into a single gated Transformer block. GAP [R]

Hi everyone! I've been experimenting with an alternative Transformer architecture called GAP (Gated Attention Projection). The main idea is to combine attention and MLP-style processing into a single gated block, instead of using the standard separate…

15s
0Read More

Options for Converting Low Voltage Landscape Lighting to Smart Lighting

I have a VOLT CTR-150-SS-R2 low voltage transformer with a VAC-TIMER3 Timer/Photocell Combination installed by a previous homeowner. I would like to upgrade it to a smart system where I can have more control over when the lights turn on and off.…

15s
0Read More

Understanding Transformer Autograd by Building It Manually in PyTorch [P]

I’ve uploaded a minimal, self-contained implementation of manual autograd for a transformer-based classifier in PyTorch. It can help build intuition for what autograd is doing under the hood and is a useful hands-on reference for low-level differentiation…

15s
Accelerating Dropless MoE Training in JAX with NVIDIA Transformer Engine
🖼️
0Read More

Accelerating Dropless MoE Training in JAX with NVIDIA Transformer Engine

NVIDIA Technical Blog·Tanya Lenz·29 days ago
#YUMaVPtz
#developer#remat#offload#expert#training

Mixture of experts (MoE) has become one of the defining architectural trends in large-scale AI model training. DeepSeek, Qwen, and Mixtral are examples of MoE…

15s
Overfitted a 900KB Transformer to Compress a 100MB CSV into 7MB
📰
0Read More

Overfitted a 900KB Transformer to Compress a 100MB CSV into 7MB

Hacker News·5 months ago
#8BQRotVU
#news#model#compression#file#transformer

Somewhat related is stavros's method to compress 500KB to something like 50 bytes https://www.stavros.io/posts/compressing-images-with-stable-...

15s
Introducing 1-bit and Ternary Bonsai Image 4B: Image Generation for Local Devices
🖼️
0Read More

Introducing 1-bit and Ternary Bonsai Image 4B: Image Generation for Local Devices

Hacker News·6 months ago
#PFjrFPvc

Images generated from Ternary Bonsai Image 4BToday we’re releasing Bonsai Image 4B, a family of compact image-generation models designed to run high-quality diffusion inference on local hardware: from laptops to phones.Bonsai Image 4B comes in two…

15s
Amazon's Panos Panay denies a new Fire Phone is in the works but doesn't shut the door
🖼️
0Read More

Amazon's Panos Panay denies a new Fire Phone is in the works but doesn't shut the door

Tom's Guide·Scott Younker·6 months ago
#ZDETacd6
#amazon#openai#att#tmobile#panay

From Tomsguide RSS Feed: Amazon's Panos Panay denies a new Fire Phone is in the works but doesn't shut the door

15s