Podcast #148: LinkedIn Live – AI is a Data Problem

StorageReview has published several review pieces and a podcast based on research conducted at Oregon State University. You can find links to those articles at the bottom of this transcript. The articles have generated significant interest not only in research but also across the vendor community. The conversation then expanded to how educators use and track AI in the classroom and during testing, with a focus on how they measure knowledge and learning quality.

The discussion centers on how all this technology is coming together to create real AI outcomes, not just in the scientific research realm, which we’ll get into, but student outcomes at Oregon State, and how these components all come together to make phenomenal solutions.

Due to the overwhelming response to the initial review conducted with Christopher Sullivan, we decided to hold a live LinkedIn event to discuss advances in AI outcomes across both research and education. Our guests included:

• Christopher Sullivan – Director of research and academic computing for the College of Earth, Ocean, and Atmospheric Sciences at Oregon State University.
• Alan Bumgardner – Director of Associate planning and pathfinding at Celadon Corporation.
• Seamus Jones – Director of technical marketing engineering for compute, networking, and sustainability at Dell.

Our participants in this live event were involved in the OSU plankton research, either through direct association (Chris) or in vendor-related roles. If you haven’t read the initial coverage or listened to the follow-up podcast, you’ll find the research both interesting and relevant to today’s environmental focus.

Although the event lasts an hour, there is no marketing or sales pitch, but it is interactive and informative. If you don’t have time to listen to the podcast from start to finish, we’ve broken it down into five-minute segments so you can hop around to the sections that interest you.

00:00–05:00 — Introduction and Context Setting

• Brian Beeler opens with a technical, no-sales discussion focused on real-world AI outcomes across research and education at Oregon State.
• Panel introductions establish cross-domain expertise spanning academic computing, storage innovation, and enterprise infrastructure.
• Oregon State’s mission is outlined: advancing scientific research while improving student engagement through AI.
• Chris Sullivan highlights a long-standing involvement that bridges research and academic IT, emphasizing the practical deployment of emerging technologies.
• Early framing positions AI as a data-centric challenge rather than purely a compute problem, setting the tone for the session.

05:00–10:00 — Data Scale and AI Workload Realities

• Large-scale scientific workloads, such as plankton imaging, generate hundreds of terabytes of data per experiment and require rapid processing.
• Training represents a small portion of the total workload; inference dominates at a massive scale, often involving billions of data points.
• Temporal relevance drives infrastructure requirements, as delayed insights reduce scientific value.
• Continuous feedback loops integrate inference results back into training models, increasing accuracy over time.
• Simulation workloads remain critical, feeding data into AI pipelines and further expanding compute demands.

10:00–15:00 — Edge AI and Infrastructure Flexibility

• Shift from centralized training clusters to distributed inference at the edge, including deployments on ships and in remote environments.
• Flexible platforms, such as GPU-dense servers with high-capacity SSDs, enable multi-environment deployment.
• High-density storage allows petabyte-scale datasets to be processed locally, reducing reliance on centralized data centers.
• Real-time processing enables adaptive data collection, improving efficiency and reducing operational costs.
• Edge inferencing introduces new design considerations for ruggedization, scalability, and modularity.

15:00–20:00 — Enterprise AI Evolution and GPU Utilization

• Industry shift from early AI investments focused on training toward widespread inference-driven deployments.
• Traditional enterprise applications are becoming AI-aware, leveraging GPUs for performance gains.
• Token usage and workflow complexity are increasing significantly as agent-based and multi-step AI processes become more prevalent.
• Infrastructure must be designed for bursty, unpredictable workloads rather than steady-state demand.
• Planning for scalability and future growth is now essential, even for initial deployments.

20:00–25:00 — Data Management and RAG Challenges

• Data accessibility emerges as a primary bottleneck, with large datasets often underutilized due to poor metadata and discoverability.
• Retrieval-Augmented Generation requires meaningful metadata, not just file-level attributes.
• Scientific and enterprise environments face similar challenges in understanding and monetizing stored data.
• Redundant datasets and a lack of visibility highlight inefficiencies in current storage practices.
• Effective data curation is positioned as foundational to unlocking AI value.

25:00–30:00 — AI in Education and Secure Data Handling

• Oregon State deploys AI-driven student evaluation systems using video-based submissions and automated assessments.
• FERPA compliance and data privacy requirements drive on-premises infrastructure decisions.
• High-capacity, local storage enables secure handling of large volumes of student-generated content.
• Systems must support continuous ingestion, processing, and retention across multiple courses and timeframes.
• AI enables scalable student feedback without increasing faculty workload.

30:00–35:00 — Cloud vs On-Prem and Token Economics

• Rising cloud costs and token consumption are driving workloads back on-premises, especially for inference.
• Tokenomics becomes a key operational consideration, particularly in academic environments with budget constraints.
• Hybrid strategies emerge, using on-prem resources for experimentation and cloud for targeted, high-value tasks.
• Query optimization and token reduction techniques are becoming critical for cost control.
• Data sovereignty and compliance concerns further reinforce on-prem adoption.

35:00–40:00 — Model Optimization and Tiered AI Architectures

• Tiered model strategies leverage smaller, specialized models for routine tasks and larger models for complex queries.
• CPU and GPU collaboration enables more efficient resource utilization across workloads.
• On-prem systems allow iterative experimentation without cost penalties, improving workflow efficiency.
• Caching and memory optimization reduce recomputation and improve performance.
• Future architectures will increasingly automate model selection and orchestration.

40:00–45:00 — Infrastructure Intelligence and Operational AI

• AI is being embedded into infrastructure management, enabling predictive monitoring and automated remediation.
• On-prem AI agents monitor campus systems, identifying failures before they impact operations.
• Smaller, targeted models are proving effective for operational intelligence use cases.
• Hardware advancements, particularly in GPUs, are delivering rapid performance gains with improved efficiency.
• Power constraints are shaping system design, prioritizing efficiency over raw scale.

45:00–50:00 — Power, Cooling, and Data Center Design

• Power density and cooling are now primary constraints in AI infrastructure deployment.
• Liquid cooling enables higher performance, reduced noise, and improved energy efficiency compared to air cooling.
• Retrofitting existing data centers with liquid cooling is viable and cost-effective.
• Storage efficiency improvements allow more power to be allocated to compute resources.
• Cooling innovations, including cold plates and immersion, significantly reduce total energy overhead.

50:00–55:00 — Performance Gains and Practical Deployment

• Liquid cooling not only improves efficiency but also enables higher sustained performance through BIOS tuning.
• Reduced acoustic impact allows AI systems to be deployed in non-traditional environments.
• Universities and enterprises can achieve meaningful ROI through power savings and performance gains.
• Cooling strategy selection requires balancing complexity, cost, and operational goals.
• Real-world deployments validate the performance benefits of advanced cooling approaches.

55:00–End — Adoption Strategies and Democratizing AI

• Lowering barriers to entry is critical, with on-prem sandbox environments enabling risk-free experimentation.
• Partnerships with vendors and ISVs are essential for integrating hardware, software, and workflows.
• Programs like hardware loaner systems allow organizations to test AI infrastructure before committing.
• Pre-validated software stacks simplify deployment and accelerate time to value.
• Broad adoption depends on enabling smaller institutions to start small, iterate quickly, and scale as needed.

The following are links to previous articles related to the research conducted by Oregon State University.

AI at Sea: Oregon State Sets Sail to Study the Ocean in Real Time
Podcast #145: Why Edge AI Matters with Oregon State University
How Metrum AI and Oregon State University Are Building the New Standard for Academic Assessment