Connect with us

Innovation and Technology

AI Inference Chip Showdown

Published

on

AI Inference Chip Showdown

Introduction to AI Inference Processing

Everyone is not just talking about AI inference processing; they are doing it. Analyst firm Gartner released a new report this week forecasting that global generative AI spending will hit $644 billion in 2025, growing 76.4% year-over-year. Meanwhile, MarketsandMarkets projects that the AI inference market is expected to grow from $106.15 billion in 2025 to $254.98 billion by 2030. However, buyers still need to know what AI processor to buy, especially as inference has gone from a simple one-shot run through a model to agentic and reasoning models that can increase computational requirements by some 100-fold.

Performance Continues to Skyrocket

For seven years, the not-for-profit group MLCommons has been helping AI buyers and vendors by publishing peer-reviewed quarterly AI benchmarks. It has just released its Inference 5.0 suite of results, with new chips, servers, and models. Let’s take a look.

The New Benchmarks

New benchmarks were added for the larger Llama 3.1 405B, Llama 2 70B with latency constraints for interactive work, and a new “R-GAT” benchmark for graph models. Only Nvidia ran benchmarks for all the models. A new benchmark was also added for edge inference, the Automotive PointPainting test for 3D object detection. There are now 11 AI benchmarks managed by MLCommons.

The New Chips

AI is built on silicon, and MLCommons received submissions for six new chips this round, including AMD Instinct MI325X (launched last Fall), Intel Xeon 6980P “Granite Rapids” CPU, Google TPU Trillium (TPU v6e) which has become generally available, Nvidia B200 (Blackwell), Nvidia Jetson AGX Thor 128 for AI at the Edge, and perhaps most importantly the Nvidia GB200, the beast that powers the NVL72 rack that has data centers scrambling to power and cool.

The New Results: Nvidia

As usual, Nvidia won all benchmarks; this time, they won by a lot. First, the B200 tripled the performance of the H200 platform, delivering over 59,000 tokens per second on the latency-bounded Llama 2 70B Interactive model. The new Llama 3.2 405B model is 3.4 times faster on Blackwell. Now for the real test: is the NVL72 as fast as Nvidia promised at launch? Yes, it is thirty times faster than the 8-GPU H200 running the new Llama 405B, but it has 9 times more GPUs.

Nvidia Performance

The new Llama 3.1 405B benchmark supports input and output lengths up to 128,000 tokens (compared to only 4,096 tokens for Llama 2 70B). The benchmark tests three distinct tasks: general question-answering, math, and code generation. But when you add Nvidia’s new open-source Dynamo “AI Factory OS” that optimizes AI at the data center level, AI factory throughput can double again running Llama and thirty times faster running DeeSeek.

And, Surprise, AMD Has Rejoined the MLPerf Party!

Welcome back, AMD! The new AMD MI325 did quite well at the select benchmarks AMD ran, competing admirably with the previous generation Hopper GPU. So, for AI practitioners who know what they are doing and don’t need the value of Nvidia software, AMD MI325 can save them a lot of money. AMD also did quite well at the Llama 3.1 405B Serving benchmark (distinct from the interactive 405B benchmark mentioned previously). AMD proudly said that Meta is now using the (older) MI300X as the exclusive inference server for the 405B model.

Conclusion

Nvidia retains the crown of AI King across all AI applications. Although competition is on the horizon, AMD delivers competitive performance only when measured against the previous Nvidia GPU generation. AMD expects that the MI350, due later this year, will close the gap. However, thanks to the GB300, Nvidia will retain the lead at the GPU performance level by then. But the real issue here is that while everyone else is trying to compete at the GPU level, Nvidia keeps raising the bar at the data center level with massive investments in software, solutions, and products to ease AI deployment and lower TCO.

FAQs

  • Q: What is the forecast for global generative AI spending in 2025?
    A: Global generative AI spending is expected to hit $644 billion in 2025, growing 76.4% year-over-year.
  • Q: What is the projected growth of the AI inference market from 2025 to 2030?
    A: The AI inference market is expected to grow from $106.15 billion in 2025 to $254.98 billion by 2030.
  • Q: What is the performance of Nvidia’s B200 chip compared to the H200 platform?
    A: The B200 chip tripled the performance of the H200 platform, delivering over 59,000 tokens per second on the latency-bounded Llama 2 70B Interactive model.
  • Q: How does AMD’s MI325 chip perform compared to Nvidia’s Hopper GPU?
    A: The AMD MI325 chip competes admirably with the previous generation Hopper GPU, and can save AI practitioners a lot of money if they don’t need Nvidia’s software.
  • Q: What is the significance of Nvidia’s Dynamo "AI Factory OS"?
    A: Nvidia’s Dynamo "AI Factory OS" optimizes AI at the data center level, allowing for doubled throughput and lower TCO.
Continue Reading
Click to comment

Leave a Reply

Your email address will not be published. Required fields are marked *

Innovation and Technology

The Future of IT: How Emerging Tech Trends are Changing the Role of IT Professionals

Published

on

Emerging tech trends in workplaces are revolutionizing the way IT professionals work, with technologies like artificial intelligence, blockchain, and the Internet of Things (IoT) transforming the industry. As these technologies continue to advance, IT professionals must adapt to stay relevant. The future of IT holds much promise, but also poses significant challenges for those who are not prepared.

Changing Role of IT Professionals

The role of IT professionals is changing dramatically, with a shift from traditional technical skills to more strategic and business-focused skills. IT professionals are no longer just technical experts, but also business leaders who must understand the organization’s goals and objectives. They must be able to communicate complex technical concepts to non-technical stakeholders and provide strategic guidance on technology investments.

Technical Skills

Technical skills are still essential for IT professionals, but they must be combined with business acumen and soft skills. IT professionals must stay up-to-date with the latest technologies, including cloud computing, cybersecurity, and data analytics. They must also be able to analyze complex data sets and provide insights that inform business decisions.

Business Acumen

IT professionals must have a deep understanding of the business and its goals, as well as the ability to communicate technical concepts to non-technical stakeholders. They must be able to identify areas where technology can improve business processes and provide strategic guidance on technology investments. This requires strong business acumen, as well as the ability to think strategically and make informed decisions.

Emerging Tech Trends

Several emerging tech trends are driving the changes in the role of IT professionals, including artificial intelligence, blockchain, and the Internet of Things (IoT). These technologies are transforming the way businesses operate and require IT professionals to have new skills and expertise.

Artificial Intelligence

Artificial intelligence (AI) is one of the most significant emerging tech trends, with the potential to automate many business processes and improve decision-making. IT professionals must understand how to implement and manage AI systems, as well as ensure that they are secure and reliable. They must also be able to analyze the data generated by AI systems and provide insights that inform business decisions.

Blockchain

Blockchain is another emerging tech trend that is changing the way businesses operate. IT professionals must understand how to implement and manage blockchain systems, as well as ensure that they are secure and reliable. They must also be able to analyze the data generated by blockchain systems and provide insights that inform business decisions.

Internet of Things (IoT)

The Internet of Things (IoT) is a network of physical devices, vehicles, and other items that are embedded with sensors and software, allowing them to collect and exchange data. IT professionals must understand how to implement and manage IoT systems, as well as ensure that they are secure and reliable. They must also be able to analyze the data generated by IoT systems and provide insights that inform business decisions.

Skills Required for Future IT Professionals

The skills required for future IT professionals are changing, with a focus on strategic thinking, business acumen, and soft skills. IT professionals must be able to communicate complex technical concepts to non-technical stakeholders, as well as provide strategic guidance on technology investments.

Strategic Thinking

IT professionals must be able to think strategically, identifying areas where technology can improve business processes and provide strategic guidance on technology investments. This requires a deep understanding of the business and its goals, as well as the ability to analyze complex data sets and provide insights that inform business decisions.

Business Acumen

IT professionals must have a deep understanding of the business and its goals, as well as the ability to communicate technical concepts to non-technical stakeholders. They must be able to identify areas where technology can improve business processes and provide strategic guidance on technology investments.

Soft Skills

Soft skills, such as communication, collaboration, and problem-solving, are essential for IT professionals. They must be able to work effectively with non-technical stakeholders, as well as provide strategic guidance on technology investments. They must also be able to analyze complex data sets and provide insights that inform business decisions.

Challenges Facing IT Professionals

IT professionals face several challenges, including the need to stay up-to-date with emerging tech trends, as well as the need to develop new skills and expertise. They must also be able to communicate complex technical concepts to non-technical stakeholders, as well as provide strategic guidance on technology investments.

Staying Up-to-Date

IT professionals must stay up-to-date with emerging tech trends, including artificial intelligence, blockchain, and the Internet of Things (IoT). This requires a commitment to ongoing learning and professional development, as well as the ability to analyze complex data sets and provide insights that inform business decisions.

Developing New Skills

IT professionals must develop new skills and expertise, including strategic thinking, business acumen, and soft skills. This requires a deep understanding of the business and its goals, as well as the ability to communicate complex technical concepts to non-technical stakeholders.

Conclusion

In conclusion, the future of IT holds much promise, but also poses significant challenges for IT professionals. Emerging tech trends, such as artificial intelligence, blockchain, and the Internet of Things (IoT), are transforming the way businesses operate and require IT professionals to have new skills and expertise. IT professionals must be able to think strategically, communicate complex technical concepts to non-technical stakeholders, and provide strategic guidance on technology investments. They must also be able to analyze complex data sets and provide insights that inform business decisions.

Frequently Asked Questions (FAQs)

Q: What are the emerging tech trends that are changing the role of IT professionals?

A: The emerging tech trends that are changing the role of IT professionals include artificial intelligence, blockchain, and the Internet of Things (IoT).

Q: What skills are required for future IT professionals?

A: The skills required for future IT professionals include strategic thinking, business acumen, and soft skills, such as communication, collaboration, and problem-solving.

Q: How can IT professionals stay up-to-date with emerging tech trends?

A: IT professionals can stay up-to-date with emerging tech trends by committing to ongoing learning and professional development, as well as analyzing complex data sets and providing insights that inform business decisions.

Q: What are the challenges facing IT professionals?

A: The challenges facing IT professionals include the need to stay up-to-date with emerging tech trends, develop new skills and expertise, and communicate complex technical concepts to non-technical stakeholders.

Q: How can IT professionals develop new skills and expertise?

A: IT professionals can develop new skills and expertise by committing to ongoing learning and professional development, as well as seeking out new challenges and opportunities.

Continue Reading

Innovation and Technology

Is AI Truly Woke?

Published

on

Is AI Truly Woke?

Introduction to AI and Its Impact

AI will be the most transformative technology of our lifetimes. However, even I, a firm advocate for the good I think it will do, can see that there is a huge amount of hype and confusion around it. Some of the biggest and most powerful corporations have bet the house on selling it to us. It’s also a highly contentious subject, with many rightly concerned about its possible impact on jobs, privacy, and security.

Concerns About AI

Another frequently voiced fear is that AI will be used to create disinformation that could further political narratives or even influence our democratic choices. There are two claims made frequently – the first is that AI can be used to spread extremist beliefs and maybe even create extremists. The second is that AI output veers towards the “woke” – a term originally used by African American civil rights protesters but now most frequently used by conservatives to refer to progressive or pro-social-justice ideas and beliefs.

Does AI Have A Left-Wing Bias?

Conservative and right-wing commentators frequently make the claim that AI and the Silicon Valley culture, where it often originates from, have a left-wing bias. And it does seem that there is at least some evidence to back up these beliefs. A number of studies, including one by the University Of East Anglia in 2023 and one published in the Journal of Economic Behavior And Organization, make the case that this is true. Of course, generative AI doesn’t actually have a political opinion – or any opinions, for that matter. Everything it “knows” comes from data scraped from the web.

Understanding AI’s Data Sources

If that data happens to support a progressive consensus – for example, if the majority of climate science data supports theories that climate change is man-made – then the AI is likely to present this as true. Rather than simply presenting facts with a left-wing bias, some of the research focuses on findings that AI will just refuse to process "right-wing image generation" requests. And when prompts describe images featuring progressive talking points like “racial-ethic equality” or “transgender acceptance,” the results are more likely to show positive images (happy people, for example).

Can AI Turn Us Into Extremists?

While some researchers are concerned that AI will turn everyone into liberals, others are more worried that it will be used to radicalize people or further extremist agendas. The International Centre For Counter-Terrorism, based at The Hague, reports that terrorist groups already widely use generative AI to create and spread propaganda. This includes using fake images and videos to spread narratives that align with their values. Terrorist and extremist groups, including Islamic State, have even released guides demonstrating how to use AI to develop propaganda and disinformation.

The Role of Humans in AI Bias

Again, this is a case of humans using AI to persuade people to adopt their views rather than an indication that AI is extreme or prone to suggesting extreme ideas and behaviors. However, one inherent risk with AI is its capability to reinforce extreme views through the algorithmic echo-chamber effect. This happens when social media and news platforms use AI to suggest content based on past engagement. This often results in users being shown more of what they already agree with, creating “echo chambers,” where people repeatedly see content that mirrors their existing beliefs. If those beliefs are extreme, AI can amplify its effect by serving up similar, more radical content.

Can AI Really Influence The Way We Think?

It’s essential to remember that while AI is likely to play an increasing role in shaping the way we consume information, it can’t directly influence our beliefs. It should also be noted that AI can also help counter these threats. It can detect bias in data, for example, that could lead to biased responses, and it can find and remove extremist content from the Internet. Nevertheless, there is clearly a perception, which appears to be justified, that groups of all political affiliations will inevitably use it to try to steer public opinion.

Conclusion

Understanding where misinformation comes from and who might be trying to spread it helps us to hone our critical-thinking skills and become better at understanding when somebody (or some machine) is trying to influence us. These skills will become increasingly important as AI becomes more ingrained in everyday life, no matter which way we lean politically.

FAQs

  • Q: Can AI have a political bias?
    A: AI itself doesn’t have opinions, but the data it’s trained on can reflect biases, which it may then present as factual.
  • Q: Can AI be used to spread extremist beliefs?
    A: Yes, AI can be used to create and spread propaganda, but this is a result of human action rather than AI’s inherent properties.
  • Q: How can AI influence political opinions?
    A: AI can influence opinions indirectly by creating echo chambers where users are shown content that aligns with their existing views, potentially amplifying extreme beliefs.
  • Q: Can AI detect and remove bias or extremist content?
    A: Yes, AI can be used to detect bias in data and remove extremist content from the internet, helping to counter misinformation and radicalization efforts.
Continue Reading

Innovation and Technology

Viral Shopify CEO Mandates AI For All Employees

Published

on

Viral Shopify CEO Mandates AI For All Employees

Introduction to AI Leadership

A leaked internal memo from Shopify CEO Tobias Lütke has quietly gone viral in executive circles — and for good reason. It is arguably one of the clearest expressions to date of what CEO leadership should look like in the age of AI.

In the memo, Lütke states plainly, “Using AI effectively is now a fundamental expectation of everyone at Shopify.” Not just developers. Not just analysts. Everyone. This is not some passing trend or another bullet point on an IT roadmap. It is a cultural shift. It’s a new way of working and thinking — one that Shopify is now weaving into its performance reviews, product development cycles and company-wide expectations. In the memo, Lütke makes the following proclamations:

  • AI Proficiency Is Now Mandatory: Using AI effectively is no longer optional at Shopify — it is a baseline expectation for all employees, regardless of role. This marks a significant cultural and operational shift.
  • Non-Use of AI Requires Justification. Employees must demonstrate why AI cannot be used before requesting additional resources (e.g., more staff or time). Stagnation is framed as failure — employees are encouraged to continuously upskill and experiment with AI.
  • AI in Product Development. AI must be integrated into the early stages (prototype phase) of all GSD (get sh** done) projects. This approach is intended to dramatically accelerate learning, iteration and team collaboration.
  • Performance Accountability. Shopify is adding AI usage to its performance and peer review criteria, making AI adoption part of how employees are evaluated and rewarded, including leadership and executive teams.

The Age of AI: A New Era for Businesses

The tone of the memo is direct and urgent. Lütke asserts that failing to adopt AI now will lead to stagnation and decline, emphasizing that employees must keep climbing — or risk sliding backward. “This exemplifies what CEO leadership looks like in the Age of AI," says Paul Baier, CEO of GAI Insights and Forbes AI contributor. “CEOs must adapt to leading organizations of, say, 1,000 employees empowered with 5,000 AI assistants.”

AI Is a Business Mindset, Not a Technology Stack

What makes the Shopify memo so compelling is its bluntness. Lütke doesn’t politely suggest employees try using AI. He doesn’t offer training modules or optional tools. He says adapt or fall behind: “Stagnation is slow-motion failure. If you’re not climbing, you’re sliding.” Lütke’s memo also reflects a broader reality: AI has already shifted from experiment to expectation. The companies that thrive in this new environment will be those whose CEOs stop treating AI as an initiative and start treating it as a core operating model.

CEOs Leading the Charge

Several other business leaders are issuing similar directives — less publicized, but equally transformative:

  • Jon Moeller, CEO of Procter & Gamble, has articulated a pragmatic but ambitious vision for AI across the company’s business units, describing it as “a force multiplier for growth and productivity.”
  • Jane Fraser, CEO of Citigroup, has made AI a central pillar of the bank’s $12 billion modernization strategy, not only for internal efficiencies but as a customer experience accelerator.
  • Chip Bergh, CEO of Levi Strauss & Co., has overseen the integration of AI to sharpen demand forecasting, optimize inventory and improve supply chain agility.

What Real CEO Leadership in the Age of AI Looks Like

Executive leadership isn’t about cheerleading AI or greenlighting another pilot project. CEOs must fundamentally shift how they lead. Here’s what that looks like in practice:

  1. Mandate Use, Don’t Just Encourage Exploration. Like Lütke, CEOs must make clear that using AI isn’t optional. AI literacy and experimentation should be expected of everyone — not just the innovation team.
  2. Tie AI to Productive Output. Shopify isn’t asking teams to build AI for AI’s sake. It wants people using AI in the prototype phase of projects — the earliest stages of idea development — when speed and iteration matter most.
  3. Formalize Accountability. Follow Shopify’s lead in including AI proficiency in feedback loops — whether through peer reviews, OKRs or team dashboards.
  4. Design for Scale, Not Just Sparkle. Stop launching pilots with no path to production. Instead, build AI usage standards, define enterprise-wide tooling strategies and create “AI-as-default” policies in business workflows.
  5. Lead Culturally, Not Just Operationally. One of the most overlooked CEO responsibilities in this era is cultural modeling. Use AI in your own daily work. Talk about it publicly. Ask teams how they’re applying it.

The Emergence of the Self-Driving Enterprise

We’re fast approaching an era when leading companies will function more like self-driving enterprises — data-fueled, agentic AI orchestrated, highly automated and continuously learning and strategically adapting in real time. That won’t happen by happenstance. It will require CEOs to shift their organizations’ very metabolism. What Shopify has shown is that the path to this future isn’t paved with expensive platforms or massive headcount changes — it’s built by creating a workplace culture where AI is part of every project, every role and every decision.

Conclusion

Organizations that treat AI as a way of life will thrive. Those that relegate it to the IT department will lag — or lose altogether. The memo from Lütke is more than an internal directive. It’s a mirror for every senior executive to look into and ask: Are we organizationally adapting? Or are we merely treating AI as a science project?

FAQs

  • Q: What is the main message of Tobias Lütke’s memo to Shopify employees?
    A: The main message is that using AI effectively is now a fundamental expectation of everyone at Shopify, and it’s a cultural shift that will be woven into performance reviews, product development cycles, and company-wide expectations.
  • Q: How does Shopify plan to integrate AI into its operations?
    A: Shopify plans to integrate AI into the early stages of all GSD projects, add AI usage to performance and peer review criteria, and make AI proficiency a mandatory skill for all employees.
  • Q: What do other CEOs think about AI adoption in their companies?
    A: Other CEOs, such as Jon Moeller, Jane Fraser, and Chip Bergh, are also making AI a central part of their business strategies, using it to drive growth, improve customer experience, and optimize operations.
  • Q: What does it mean for a company to be a "self-driving enterprise"?
    A: A self-driving enterprise is a company that functions like a self-driving car, using data and AI to automate and optimize its operations, and continuously learn and adapt in real-time.
Continue Reading
Advertisement

Our Newsletter

Subscribe Us To Receive Our Latest News Directly In Your Inbox!

We don’t spam! Read our privacy policy for more info.

Trending