Yu-Gi-Oh!'s Seto Kaiba is a character who grew out of adversity. Having grown up in an orphanage, Seto had to fend for himself and his younger brother before securing their adoption into KaibaCorp CEO Gozaburo Kaiba's family. Seto then had to survive the cutthroat worlds of Kaiba family life, the KaibaCorp business and, finally, Duel Monsters.
Despite his ostentatious wealth (and personality) Kaiba actually struggled quite a bit throughout his life, even with his favorite card game, as villains constantly came out the woodwork to suck the fun out of it. Seto Kaiba came up in a dog-eat-dog world where some people are simply above others. It seems he wanted to share that experience with his wards at the Duel Academy.
Duel Academy's Dorms
Students at Duel Academy typically live in one of three dorms: Slifer Red, Ra Yellow and Obelisk Blue. However, these dorms also form a social hierarchy. Named for the Egyptian God Cards, dorm placement actually reflects on the skill of the duelists set to reside there. Slifer The Sky Dragon is a strong card (in the anime), but is often seen as the weakest of the God Cards due to its double-edged ability. Meanwhile, Obelisk the Tormentor -- the God Card Kaiba personally used in the Battle City tournament -- is considered the strongest of these cards.
Slifer Red Dorm
Similar to the card's standing, the residents of Slifer Red sit at the bottom of the school hierarchy. These duelists are the least skilled students at the academy, constantly a loss away from dropping out entirely. Due to their poor standing, their dorm has next to no amenities, the idea being that, without distractions, they can focus on improving their game. But cramped, four-person rooms and spoiled prison food seem a bit much to inflict on tuition-paying students.
Ra Yellow Dorm
Ra Yellow is the dorm for students with plenty of potential. While not generally considered aces at the game, these students put in a solid effort to get good grades and duel skillfully. Bastion Misawa is among the notable members of this dorm, a student at the top of his class. While he earns the right to join the Obelisk Blue dorm early on in the series, he rejects the promotion because he wants to prove himself as a better duelist than Jaden.
Obelisk Blue Dorm
The cream of the crop are the students of Obelisk Blue, duelists whose lives at the school are a far cry from those suffering in Slifer Red. These students stand head and shoulders above everyone else, but they need a year to prove themselves -- freshmen aren't typically placed in the dorm.However, exceptions can be made as Jaden's rival, Chazz Princeton, got in as a first-year. In correlation with their standing, these students live in fancier dorms tricked out with all sorts of commodities for students to enjoy, along with gourmet food readily available. Zane Truesdale, a member of the Obelisk Blue dorm, stood out as one of the best duelists at Duel Academy before he graduated.
The Dorms Are a Caste System
What this system creates, unintentional or not, basically amounts to a caste system. High-ranking students look down at those below them, taking the Kaiba-esque outlook of equating skill with worth as a person. Chazz Princeton, for example, early in Yu-Gi-Oh! GX's run looked down on students less skilled than him, thinking his placement in Blue made him worthy of something they weren't. On top of that, over the course of GX there are teachers who put down Slifer Red students, in some cases finding excuses to expel them. Dr. Crowler is the worst offender, with his schemes to finish Jaden off through Chazz and Boneparte, staking his enrollment on duels.
The major upside to this system is that students aren't actually locked into their dorm. There are opportunities for students to go up in rank, and conversely, fall if they perform poorly. Special exams are a chance for students to put their fate in their own hands. By going up against strong opponents and coming out the other side victorious, students are able to overcome whatever hurdles they might've started with, using their own strength. And it's not uncommon -- throughout the series, there are students like Syrus Truesdale who manage to progress through all three dorms. Teaching students to overcome bad odds with wits and skill... surprisingly, Kaiba gave his young wards the tools to level up in life as he had.
Jaden's existence in the school is an anomaly to Kaiba's system. As a skilled duelist with no regard for rank or personal pride in how others see him, Jaden totally degrades the entire hierarchy of duelists, so much so that, by the end of the series, students actually want to be in the Red dorm. While Jaden isn't a model student by any means, often sleeping through class, his dueling prowess speaks for itself. Jaden ends up staying in the Slifer Red dorm throughout his time at Duel Academy but, by the end, it doesn't even matter -- he'd already toppled the system.