The City of Kelowna once asked its AI chatbot which beaches had lifeguards.
The answer came back confidently wrong.
The bot listed lifeguards at beaches that did not have them, because somewhere, deep in the city’s website, sat an old PDF from when Kelowna’s beaches did have lifeguards.
Nobody had touched that document in years, but the AI found it anyway.
You could bring up Jurassic Park and “life finds a way,” but that’s a whole other kettle of fish with AI and the human thoughts and emotions of it all. It is, of course, a pretty telling anecdote.
“It’s a funny example,” James McGregor, chief technology officer (CTO) at the City of Kelowna, told at a CIOCAN Peer Forum session on moving from AI pilots to enterprise intelligence, “but it’s also quite critical a mistake.”
In addition to a chuckle, it’s also a rather on-point example of why AI transformation in the public sector is all about the plumbing.
Canadian municipal technology leaders carrying decades of accumulated technical debt need to know how to incorporate AI without ancient PDFs eating your credibility.
To explain how Kelowna does it, especially budget-wise, McGregor turns to baseball.
He likens their approach to ‘small ball,’ or hitting singles and doubles. In practice, this means targeted deployments in areas where data is clean enough to trust, processes consistent enough to automate, and the blast radius of a mistake is manageable.
Kelowna’s chatbot program began as a way to answer common service questions outside office hours. It now handles roughly 180,000 questions a year, with about a 40% conversion rate on interactions that require no human intervention. It’s certainly not a moon shot, because it’s a measurable, repeatable win that buys the credibility to go after bigger things.
The approach McGregor and his team took has lessons for any tech leader who has ever had to tell a CEO that the foundation needs work before the bigger picture stuff can land.
The 30-year-old problem sitting in your infrastructure
Kelowna’s AI journey started earlier than most people might expect. McGregor’s predecessor began exploring AI-assisted citizen services back in 2018, long before LLMs were a board-level conversation.
City offices were open from 8 a.m. to 4 p.m., but citizens need information around the clock. Chatbots could answer service questions and requests after hours. These small, focused applications were designed to extend the city’s reach without additional hiring that might be needed otherwise. proportional headcount growth.
What McGregor inherited when he joined about 18 months ago was both a head start and a reckoning.
“We’re dealing with these cutting-edge elements of AI,” he said. “But we’ve got this 30-year-old ColdFusion stuff sitting out in the wild.”
Modernizing legacy infrastructure while also deploying new AI capabilities is a problem technology leaders at mid-market organizations across Canada are grappling with.
Boards and c-suites go all Marco Polo and “discover” AI, but the infrastructure is just not there.
McGregor’s approach is to run both tracks simultaneously rather than sequence them. Deploy AI in areas where the data cooperates, and put the foundational modernization to work in parallel.
The City is in the middle of a significant enterprise resource planning (ERP) implementation, and data governance work is being tied directly to that project.
Theory, meet practice.
“Part of my challenge is moving from the theoretical to the practical,” McGregor said. “What does it actually mean? How does that show up in a meaningful way?”
More technology leaders should be asking this question out loud to their teams.
Playing small ball, and why that is a strategy
In a session nominally about scaling AI, McGregor spent a fair amount of time making the case for not scaling.
At least not yet, and not everything everywhere all at once.
He frames it as being kind to your future self. Large, decentralized AI deployments without proper governance creates the exact technical debt you’re trying to escape.
For example, agents end up interacting with other agents in ways that produce unforeseen consequences, or processes that become embedded in the business in ways that IT owns but can’t sustainably support.
Continuing with the baseball metaphor, he says, “Stepping up and hitting the home runs creates tremendous risk, and it’s maybe not aligned with where we’re at with this technology.”
The success of Kelowna’s chatbot program emphasizes this point. The city isn’t ready to declare victory on citizen service automation quite yet, but the existing deployment acts as a proof of concept for a larger city services hub project.
It’s akin to a 311 service that Kelowna doesn’t currently have. The chatbot data informs what that hub should look like, where demand concentrates, what questions need human judgment, and where the handoff points should be.
Robbie Beyer, director of data science and AI at RSM, worked with Kelowna on the implementation.
As AI models continue to advance in capability, what will differentiate organizations is the organizational data those models have access to.
“They’re being trained on data that’s on the open internet, but they’ve never seen your organization-specific data,” Beyer said. “That’s going to make you different than the organization down the street. So as you double down and focus on getting that data foundation right, that will continually stay increasingly valuable as the features continue to advance.”
For public-sector organizations in particular, that foundational work is the competitive moat.
The city that has clean, well-governed data about its permitting process, citizen service patterns, and operational workflows will find value in future AI capabilities. Compare this to the city that chased every shiny object and now has a governance problem layered on top of a data problem layered on top of a legacy infrastructure problem, like a civic turducken.
What the private sector keeps getting wrong
The failure rate question came up during the session, and McGregor did not sidestep it. He puts the failure rate of AI projects at roughly double that of other technology initiatives.
The research backs up his estimates. In partnership with Lenovo, IDC found that 88% of observed proof-of-concepts don’t pass through to widescale deployment. An MIT report from 2025 found that 95% of organizations had a ROI of zero, despite enterprise investment in GenAI hitting $30-40 billion.
His theory is that they are nebulous by nature. The definition of success is fuzzy, and its value is hard to measure. Not to mention that process changes often end up being far more involved than originally thought.
“Having a partner who’s been through it [and] can guide you through that process is really key,” McGregor said. “But you don’t want to give up the strategic value of the work. You don’t want to give up the strategic understanding, and you want to have your teams involved in those types of things.”
Kelowna’s model is a hybrid, with partners for when they don’t have the expertise in-house or when they need to move fast, and internal ownership of strategy and architecture decisions, staff augmentation for specialized skills that cannot be hired in a reasonable timeframe.
McGregor was candid that municipal hiring timelines are not designed for the pace of technology change. Planning six months ahead for a role is not realistic timing when the need may have evolved by the time the new hire is signing onboarding paperwork.
The broader lesson is one the private sector is slower to absorb than it should be.
Transformation programs, with their promises to leap from legacy to leading-edge in a single initiative, consistently underperform compared to targeted pilots that build institutional knowledge alongside the technology itself.
Every failed pilot is a data management audit in disguise. The organizations that treat it that way are the ones that get somewhere.
Final shots
- 180,000 citizen questions were handled by Kelowna’s chatbot last year, 40% without human intervention, and the municipality still calls it a proof of concept.
- A 30-year technology footprint isn’t going to be a roadblock in deploying AI. You just need to start where the data cooperates.
Digital Journal is the national media partner for the CIO Association of Canada.