Sign in Subscribe

Building a Custom Action Item Tool: Beyond Microsoft Teams Copilot's Limitations

Teams Copilot struggles with long meetings, 30-day history limits, and inconsistent formatting. Learn how to build a custom Azure solution using durable functions and OpenAI that processes any meeting length with unlimited history and team-specific templates.

Building a Custom Action Item Tool: Beyond Microsoft Teams Copilot's Limitations

Why Standard Action Item Tools Fall Short

If you've used Microsoft Teams Copilot to capture action items during meetings, you know it can be a game-changer. But you've probably also hit its limits. Based on Microsoft's documentation, Copilot struggles with meetings longer than two hours, can't process historical data beyond 30 days, and lacks the ability to format action items consistently across your organization.

As someone who's led strategic quality programs across Azure and orchestrated executive forums, I've seen these limitations impact teams when every action item matters. When mission-critical decisions are being made, you need a tool that scales with your needs.

The Challenges with Microsoft Teams Copilot

Before diving into building a custom solution, let's understand what we're trying to solve:

  1. Time Limitations: Copilot struggles with meetings over two hours. For enterprise architecture reviews or quarterly planning sessions that run longer, this is a significant problem.
  2. Format Inconsistency: Each action item gets captured differently, making tracking and follow-ups harder across multiple meetings.
  3. Limited Historical Context: Copilot can only access 30 days of chat history, making it difficult to connect current action items with ongoing initiatives.
  4. Template Rigidity: No ability to customize action item formats based on team or project needs.

Having worked with Fortune 500 clients during my time at Microsoft's AzureCAT, I've seen how these limitations affect enterprise-scale operations where consistency is crucial.

Building a Custom Action Item Solution

Let's build a scalable solution using Azure that removes these limitations. I'll walk through the complete architecture, drawing on my experience delivering cloud solutions for global enterprise clients.

Azure Architecture Overview

The solution uses these Azure components to create a reliable, scalable pipeline:

  1. Microsoft Graph API: Accesses Teams meeting recordings and transcripts stored in OneDrive/SharePoint
  2. Azure Functions: Processes transcripts and coordinates workflow steps
  3. Azure OpenAI Service: Extracts and formats action items using custom prompts
  4. Azure Cosmos DB: Stores action items with indefinite history retention (beyond Copilot's 30-day limit)
  5. Azure Cognitive Search: Enables context-aware search across all past action items
  6. Azure Key Vault: Secures API keys and credentials
  7. Azure App Service: Hosts the web interface for teams to interact with the tool

Here's how data flows through this pipeline:

The Processing Pipeline

The heart of the solution is a reliable processing pipeline that handles meetings of any length using Azure Durable Functions:

// Durable Function orchestrator for processing Teams recordings
[FunctionName("ProcessMeetingOrchestrator")]
public static async Task<List<ActionItem>> RunOrchestrator(
    [OrchestrationTrigger] IDurableOrchestrationContext context)
{
    var meetingId = context.GetInput<string>();
    
    try
    {
        // Step 1: Get recording from Teams
        var recording = await context.CallActivityAsync<Recording>(
            "GetTeamsRecording", meetingId);
        
        // Step 2: Get or create transcript
        var transcript = await context.CallActivityAsync<string>(
            "GetTranscript", recording);
        
        // Step 3: Chunk transcript for parallel processing
        var chunks = await context.CallActivityAsync<List<string>>(
            "ChunkTranscript", transcript);
        
        // Step 4: Process chunks in parallel (this is where durable functions shine)
        var tasks = chunks.Select(chunk => 
            context.CallActivityAsync<List<ActionItem>>("ExtractActionItems", chunk));
        
        var results = await Task.WhenAll(tasks);
        
        // Step 5: Merge and deduplicate results
        var allItems = results.SelectMany(x => x).ToList();
        var mergedItems = await context.CallActivityAsync<List<ActionItem>>(
            "MergeAndDeduplicate", allItems);
        
        // Step 6: Store in database and search index
        await context.CallActivityAsync("StoreActionItems", mergedItems);
        
        return mergedItems;
    }
    catch (Exception ex)
    {
        // Durable functions handle retries automatically
        throw new Exception($"Failed to process meeting {meetingId}: {ex.Message}");
    }
}

// HTTP trigger to start the orchestration
[FunctionName("ProcessMeeting")]
public static async Task<IActionResult> HttpStart(
    [HttpTrigger(AuthorizationLevel.Function, "post")] HttpRequest req,
    [DurableClient] IDurableOrchestrationClient starter)
{
    var meetingId = await req.ReadAsStringAsync();
    
    string instanceId = await starter.StartNewAsync(
        "ProcessMeetingOrchestrator", null, meetingId);
    
    return starter.CreateCheckStatusResponse(req, instanceId);
}

Durable functions solve several problems here:

  1. Long-running processes: Some meetings generate hours of transcript that take 10+ minutes to process
  2. Parallel processing: We can process multiple transcript chunks at the same time
  3. Automatic retries: If OpenAI API calls fail, durable functions retry them
  4. State management: The function remembers where it stopped if something fails
  5. Template System

Create a flexible template system that allows teams to standardize how action items appear:

// Template system with factory method
public static class TemplateFactory
{
    public static Dictionary<string, Template> CreateDefaultTemplates()
    {
        return new Dictionary<string, Template>
        {
            ["executive"] = new Template
            {
                Format = "{date} | {owner} | {action} | {dueDate} | {status}",
                Required = new[] { "owner", "dueDate" }
            },
            ["development"] = new Template
            {
                Format = "{issue} | {owner} | {action} | {linkedTicket}",
                Required = new[] { "owner", "linkedTicket" }
            }
        };
    }
}
  1. Historical Context Integration

Unlike Copilot's 30-day limit, our solution maintains a database of past action items for reference:

// Action item model and search functionality
public class ActionItem
{
    public string Description { get; set; }
    public string Owner { get; set; }
    public DateTime? DueDate { get; set; }
    public string Status { get; set; }
    public string[] Keywords { get; set; }
}

[FunctionName("FindRelatedActions")]
public static async Task<List<ActionItem>> FindRelatedActions(
    [ActivityTrigger] ActionItem newActionItem,
    [CosmosDB(databaseName: "ActionItems", collectionName: "Items")] 
    CosmosClient cosmosClient)
{
    // Search historical actions using keywords
    var keywords = ExtractKeywords(newActionItem.Description);
    return await SearchActionItemDatabase(keywords, cosmosClient);
}

private static async Task<List<ActionItem>> SearchActionItemDatabase(
    string[] keywords, 
    CosmosClient cosmosClient)
{
    var container = cosmosClient.GetContainer("ActionItems", "Items");
    var keywordQuery = string.Join(" OR ", keywords.Select(k => $"CONTAINS(c.description, '{k}')"));
    
    var query = new QueryDefinition($"SELECT * FROM c WHERE {keywordQuery}");
    var iterator = container.GetItemQueryIterator<ActionItem>(query);
    
    var results = new List<ActionItem>();
    while (iterator.HasMoreResults)
    {
        var response = await iterator.ReadNextAsync();
        results.AddRange(response);
    }
    
    return results;
}
  1. Action Item Intelligence

The real power comes from pattern recognition across meetings:

// Meeting types and pattern analysis
public enum MeetingType
{
    Standup,
    Planning, 
    Review,
    Retrospective
}

public class Pattern
{
    public string Format { get; set; }
    public int Frequency { get; set; }
}

[FunctionName("SuggestActionFormat")]
public static async Task<string> SuggestActionFormat(
    [ActivityTrigger] FormatRequest request,
    [CosmosDB(databaseName: "ActionItems", collectionName: "Items")] 
    CosmosClient cosmosClient)
{
    // Analyze how this meeting type typically formats actions
    var patterns = await AnalyzePastActionPatterns(
        request.MeetingType, 
        request.PastActions, 
        cosmosClient);
    
    return RecommendFormat(patterns);
}

public class FormatRequest
{
    public MeetingType MeetingType { get; set; }
    public List<ActionItem> PastActions { get; set; }
}

private static async Task<List<Pattern>> AnalyzePastActionPatterns(
    MeetingType meetingType,
    List<ActionItem> pastActions,
    CosmosClient cosmosClient)
{
    var container = cosmosClient.GetContainer("ActionItems", "Items");
    
    var query = new QueryDefinition(
        "SELECT c.format, COUNT(1) as frequency FROM c WHERE c.meetingType = @type GROUP BY c.format")
        .WithParameter("@type", meetingType.ToString());
    
    var iterator = container.GetItemQueryIterator<Pattern>(query);
    var patterns = new List<Pattern>();
    
    while (iterator.HasMoreResults)
    {
        var response = await iterator.ReadNextAsync();
        patterns.AddRange(response);
    }
    
    return patterns.OrderByDescending(p => p.Frequency).ToList();
}

Implementation Strategy

Based on my experience leading infrastructure projects and building technical enablement programs, here's the implementation roadmap:

Phase 1: Core Functionality

  1. Build transcript processing engine using Azure Cognitive Services
  2. Create action item extraction logic with customizable pattern matching
  3. Develop basic template system with 3-5 starter templates
  4. Set up secure storage for transcripts and extracted items

Phase 2: Intelligence Layer

  1. Implement cross-meeting pattern recognition
  2. Add suggestion engine for action item formats
  3. Build historical context search
  4. Develop owner recognition and tracking

Phase 3: Integration & Scale

  1. Connect with ticketing systems (Azure DevOps, Jira)
  2. Add automatic follow-up reminders
  3. Build reporting dashboards
  4. Implement enterprise-grade access controls

Real-World Application Example

Let me walk through how this works in practice, drawing from my experience with enterprise clients:

When I led the $1.5M IT transformation project for YES Prep Public Schools, we conducted weekly 3-hour planning sessions. A tool like this would have changed our process:

  1. Meeting Ends: Teams saves the recording to OneDrive, triggering our Logic App
  2. Recording Access: The durable function gets the recording via Microsoft Graph API
  3. Transcript Processing: Azure Functions chunk the 3-hour transcript into manageable pieces
  4. Parallel Extraction: Multiple chunks get processed at once through Azure OpenAI, each following our education project template
  5. Context Matching: Cognitive Search finds related action items from previous weeks and connects them
  6. Results Delivery: The formatted action items appear in Teams chat within 10 minutes

The difference? Instead of spending 2-3 hours after each session manually writing up action items, we'd have had a formatted, contextualized list immediately. Over the 18-month project, this would have saved approximately 150 hours of administrative work.

Cost Analysis and ROI

Based on my experience implementing similar solutions, here's the realistic cost breakdown for this Azure architecture:

Monthly Azure Costs (estimated for a 500-person organization):

  • Azure Functions: $50-100/month (depending on meeting frequency)
  • Azure OpenAI Service: $200-400/month (based on transcript processing volume)
  • Azure Cosmos DB: $100-200/month (for action item storage)
  • Azure Cognitive Search: $150-300/month (for historical search)
  • Azure Media Services: $100-200/month (for transcription)
  • Other services (Logic Apps, Blob Storage, Key Vault): $50-100/month

Total estimated cost: $650-1,300/month

ROI Calculation:

  • Average time saved per meeting: 30 minutes of post-meeting cleanup
  • Meetings per week per team: 5
  • Average hourly rate for knowledge workers: $75
  • Time savings per team per week: 2.5 hours × $75 = $187.50
  • For 20 teams: $3,750/week or $195,000/year in time savings

The solution pays for itself within the first month for most organizations.

Building vs. Buying

As with any technical solution, you need to weigh building versus buying. Here are the requirements for building this tool:

Technical Skills Required:

  1. Azure Architecture: Experience with Azure Functions, Logic Apps, and Cosmos DB
  2. LLM Integration: Understanding of Azure OpenAI Service and prompt engineering
  3. Bot Development: Azure Bot Framework and Teams integration
  4. Search Implementation: Azure Cognitive Search setup and indexing
  5. Security: Azure Key Vault and enterprise-grade access controls

Development Timeline:

  • Phase 1 (Core): 8-12 weeks
  • Phase 2 (Intelligence): 6-8 weeks
  • Phase 3 (Integration): 4-6 weeks
  • Testing & Deployment: 4-6 weeks

Total: 22-32 weeks with a team of 3-4 developers

If your organization doesn't have these capabilities in-house, partnering with a development team is worth considering. The ROI comes from three areas:

  1. Time Savings: Cut manual action item formatting and reduce meeting follow-up time by 60%
  2. Accuracy Improvement: Reduce missed action items by 40% through consistent AI processing
  3. Context Enhancement: Improve follow-through by connecting related items across meetings and projects

Enterprise Considerations

Having worked with Fortune 500 clients, I've learned these considerations are crucial for enterprise adoption:

Security and Compliance

csharp

// Security configuration for enterprise deployment
public class SecurityConfig
{
    public EncryptionSettings Encryption { get; set; } = new EncryptionSettings
    {
        AtRest = true, // All data encrypted in Cosmos DB and Blob Storage
        InTransit = true, // HTTPS/TLS for all communications
        KeyManagement = "Azure Key Vault" // Centralized key management
    };
    
    public AccessSettings Access { get; set; } = new AccessSettings
    {
        Authentication = "Azure AD", // Single sign-on integration
        Authorization = "Role-based", // Granular permissions per team
        AuditLog = true // Complete audit trail of all actions
    };
    
    public ComplianceSettings Compliance { get; set; } = new ComplianceSettings
    {
        DataResidency = "Configurable", // Meet regional data requirements
        Retention = "Policy-based", // Automatic cleanup based on org policies
        Gdpr = true // GDPR compliance features
    };
}

public class EncryptionSettings
{
    public bool AtRest { get; set; }
    public bool InTransit { get; set; }
    public string KeyManagement { get; set; }
}

public class AccessSettings
{
    public string Authentication { get; set; }
    public string Authorization { get; set; }
    public bool AuditLog { get; set; }
}

public class ComplianceSettings
{
    public string DataResidency { get; set; }
    public string Retention { get; set; }
    public bool Gdpr { get; set; }
}

Scalability Planning

The architecture scales horizontally across Azure regions. For global organizations, deploy the solution in multiple Azure regions with data replication for optimal performance.

Change Management

Based on my experience leading technology adoption at Microsoft, plan for:

  1. Pilot Program: Start with 2-3 high-value teams
  2. Template Development: Work with teams to create their preferred formats
  3. Training: 2-hour sessions on using the Teams bot and web interface
  4. Feedback Loop: Weekly check-ins during the first month

Next Steps

If you're interested in implementing a solution like this in your organization:

  1. Assessment Phase: Document your specific action item tracking pain points and current meeting patterns
  2. Template Design: Work with key teams to create standardized formats for different meeting types
  3. Integration Planning: Identify which systems (Azure DevOps, Jira, ServiceNow) need to connect with your action item tracking
  4. Pilot Program: Start with a high-value team that conducts lengthy meetings and has clear success metrics
  5. Architecture Review: Validate the Azure components fit your organization's existing cloud strategy

Having worked with technology teams across industries, I've seen how the right tool can transform meeting productivity. Building a custom action item tool isn't just about capturing tasks—it's about creating an organizational memory that makes follow-through more effective and connects work across time and teams.

Conclusion

Microsoft Teams Copilot offers a great starting point for capturing action items, but enterprises with complex needs will quickly hit its limitations. By building a custom Azure-based solution that addresses time constraints, formatting inconsistency, historical context, and template rigidity, you can create a tool that truly scales with your organization's needs.

The approach outlined here leverages my experience in cloud architecture and enterprise solutions to deliver a tool that works for meetings of any length, maintains critical context across unlimited history, and helps teams standardize how they track and complete action items. The combination of Azure's scalable infrastructure and custom LLM prompting creates a solution that adapts to your organization's specific needs rather than forcing you to adapt to a tool's limitations.

The investment in building this custom solution pays for itself quickly through time savings and improved follow-through, while providing the flexibility to evolve with your organization's changing needs.

If you're interested in discussing how a solution like this could work in your organization, contact me for a consultation on architecture design and implementation planning.