Technology Assessment Questionnaire

We use this questionnaire during the assessment phase of our fractional CTO engagements. It surfaces the technical risks, scaling constraints, and governance gaps that determine whether your value creation thesis delivers. Expect the full review to take 2–4 weeks; even a quick pass highlights where you are exposed.

Artificial Intelligence & Machine Learning: Strategic Direction

• Has your organization built an AI/ML roadmap that actually connects to what you’re trying to sell?
• Does product management think about AI/ML when planning features, or is it an afterthought?
• How well do you understand your dependency risks on AI/ML vendors or cloud providers? This matters more than you think.

Artificial Intelligence & Machine Learning: Implementation

• Have you identified the right data repositories for AI applications and set up proper governance?
• Do you have standardized processes for taking models from testing to production?
• Are production models monitored for performance degradation and data drift?
• Do you have processes for detecting bias, fairness issues, and ethical risks in your models?
• Are there safeguards protecting your AI/ML infrastructure and data from security threats?
• Has model inference been optimized for latency, scalability, and cost efficiency?
• Are you using modern, scalable ML infrastructure and tools effectively?
• Do your AI/ML operations comply with relevant regulations such as GDPR or HIPAA?
• What mechanisms control costs across AI/ML development and operations?
• Have you defined protocols for retraining models based on performance thresholds or data changes?
• Does your team understand and manage ethical risks from AI deployments?
• Are third-party models and tools used with appropriate governance and risk awareness?
• Do AI/ML rollouts use experimentation and progressive deployment strategies?

Artificial Intelligence & Machine Learning: Competitive Advantage

• Are data scientists, engineers, and domain experts collaborating effectively and sharing knowledge?
• Are large language models used strategically and grounded in validated data?
• Can your models provide clear explanations and interpretability to internal teams and external stakeholders?
• Are AI/ML capabilities improved based on customer usage patterns and feedback?

Growth Capacity: Horizontal Scaling

• Are load balancers or ELBs used to distribute traffic across multiple instances?
• Is session state stored client-side or in a separate layer rather than on application servers?
• Have you implemented read/write separation or another horizontal database scaling pattern?
• Are caching layers deployed? If yes, describe the strategy and implementation.
• Are authentication and authorization services centralized and available across all environments and regions?
• What strategies are used for CDN and browser caching?

Growth Capacity: Functional Decomposition

• Are different functions (auth, registration, checkout) separated across services?
• Is data partitioned across separate databases, and does each service own its data?
• Are services sized appropriately based on availability needs, change frequency, dependencies, and team structure?

Growth Capacity: Data-Based Partitioning

• Are application instances assigned to handle specific subsets of similar data such as customers or products?
• Are database instances allocated to manage only specific subsets of comparable data?
• What’s your approach to multi-tenancy or shared-tenancy at the service and data layers?
• How are geographic data residency requirements handled, and are they required by GDPR or similar regulations?

Failure Isolation (Fault Zones)

• Are cross-service communications asynchronous only?
• Do you maintain data isolation by domain and service?
• Does your architecture enforce boundaries between domains and concerns?

Disaster Recovery

• Are there single points of failure where one component going down causes a widespread outage?
• Can your data layer survive logical corruption through backups and point-in-time recovery?
• What strategies use multiple availability zones or regions for disaster recovery?
• Are physical data centers located in geographically stable, low-risk areas?

Cost Optimization

• Does your system use database stored procedures for business logic?
• Are presentation, business logic, and data layers deployed on separate hardware or virtual machines?
• What mechanisms enable scaling up or down based on demand? Are auto-scaling capabilities in use?
• What third-party technologies are part of the stack, and how do they impact delivery speed, costs, and performance?
• For on-premises infrastructure, what hardware backs each tier? For cloud deployments, which instance types run core workloads?
• Is virtualization used? If so, what goals drive that decision?
• Are physical data centers in cost-effective locations?
• What traffic flows through firewall and web application firewall layers?
• How complex and expensive would it be to migrate to a different public cloud provider?
• Does custom code exist per customer? How many versions are deployed and maintained simultaneously?
• What framework guides platform service adoption today, and how will that evolve?

Product Management Function

• Does a dedicated product management function exist with decision rights over feature additions, delays, or removals?
• Does product management own business outcomes?
• Are success metrics, OKRs, KPIs, or customer feedback loops defined and used to drive prioritization?
• Do you run iterative validation cycles to confirm market value and achieve goals?
• Are leading and lagging metrics defined for major initiatives, and how are they tracked, validated, and revisited?

Product Development Lifecycle Management

• Are leading and lagging indicators defined for major initiatives, and how are they monitored?
• Do you use relative estimation techniques to size features and user stories?
• Do you track velocity to improve delivery predictability?
• Are sprint metrics, KPIs, and retrospectives used to assess progress and effectiveness?
• How do you measure engineering productivity, and who owns the improvement plan?
• Do you compare estimates versus actuals?
• Is there a clear definition of done that measures customer adoption instead of just code deployed?

Engineering Practices

• Can you identify and use production hotfix branches for rapid deployment?
• Do you use a feature branch strategy, and can a single engineer block a release?
• How do you provision new servers, environments, or regions within your cloud infrastructure?
• Have you documented coding standards, and how are they enforced or automated?
• Do engineers perform code reviews with defined criteria or automated validation?
• Is open-source license management tracked, and is it automated?
• Do engineers write unit tests for their code?
• Does automated testing cover at least 75% of the codebase?
• Do you practice continuous integration?
• Are load and performance tests run before exposing features to large user populations, or integrated into the pipeline?
• Do you release small incremental changes frequently instead of infrequent big bang releases?
• Do you practice continuous deployment?
• Have you implemented containerization (like Docker) and orchestration platforms (such as Kubernetes)?
• Are feature flags used so features can be toggled independent of deployments?
• Can you roll back deployments quickly through configuration toggles, tested database scripts, and incremental schema changes?
• What framework classifies technical debt, and how is it tracked?
• Is a fixed percentage of each sprint allocated to debt paydown?
• How do production issues, bug backlogs, and technical debt flow back into planning?
• Do you run architecture design sessions that include engineering and operations?
• Are architecture principles documented and consistently followed?
• Does an architecture review board evaluate major features?

Operations Practices

• What’s your logging approach? Where are logs stored, are they centralized and searchable, and who has access?
• Do you monitor user experience metrics to detect issues?
• Do you track infrastructure-level metrics that pinpoint root causes?
• Are synthetic monitors deployed against critical transaction paths so you know about issues before customers do?
• Are production incidents centrally tracked with adequate detail?
• Are root causes distinguished from symptoms and captured in a central system?
• Do you classify incident severity, and how are major incidents escalated to leadership?
• Are alerts routed immediately to the right owners and subject matter experts?
• Is there a single location where all production changes—code and infrastructure—are logged?
• Do you run postmortems for major incidents with action items tracked to completion?
• Do you measure time to fully resolve incidents?
• Is system availability measured by actual customer impact?
• Do you hold service quality reviews that cover complaints, incidents, SLAs, and postmortems?
• Do you run monthly or quarterly operational reviews that surface architectural improvements?
• Do you know the remaining capacity in your infrastructure and the runway before constraints hit?
• How are customer-reported issues routed from support to engineering?
• How do you test failure scenarios, including chaos engineering?

Team Capabilities and Structure

• Do architects have experience in both development and infrastructure?
• What processes support onboarding and skill development?
• Are teams regularly assessed and upgraded, and how are underperforming members handled?
• Are teams organized around services or product features in independent squads?
• Can sprint teams deliver independently with reasonable timelines?
• Do teams have the full skill set necessary to hit their goals?
• Does leadership treat system availability as a product feature by investing in debt paydown and scale readiness?
• Have architects designed for graceful degradation using proven scalability patterns?
• Is the ratio of engineers to QA specialists appropriate?
• Do you think strategically about hiring across experience levels and geographies?
• What percentage of QA work is automated?

Security

• Has your organization published and adopted a comprehensive information security policy?
• Has a specific person been appointed with overall responsibility for information security?
• Are security responsibilities clearly defined across teams?
• Are security goals and objectives communicated across the company?
• Is there an ongoing security awareness and training program for all staff?
• Is there a complete inventory of data assets with assigned owners?
• Has a data classification scheme been established based on legal, regulatory, and business requirements?
• Is an access control policy enforced so users only access resources required for their roles?
• Are access rights revoked immediately when employees or partners depart?
• Is multi-factor authentication required for critical systems?
• Is source code access restricted to necessary personnel?
• Are development and test environments separated from production?
• Are network security scans performed regularly with remediation prioritized by business risk?
• Are application security scans or penetration tests performed regularly and after major changes?
• Is all sensitive data encrypted in transit?
• Is security functionality tested during development?
• Are security requirements incorporated into development standards?
• Is the incident response plan documented and tested at least annually?
• Are encryption technologies used in compliance with agreements, laws, and regulations?
• Is there a formal process for assessing and prioritizing security risks?
• Have intrusion detection and prevention systems been deployed?

How to Use This Questionnaire

We walk through this framework during the assessment phase—typically Step 2 in our process. It takes 2–4 weeks depending on complexity, but once it’s complete you know exactly what’s broken, what’s risky, and what needs to be fixed before the next board meeting.

Frequently Asked Questions

Q: What is a technology assessment questionnaire and how is it used?

A: A technology assessment questionnaire is a comprehensive diagnostic framework used during fractional CTO engagements to uncover technology risks that impact exit multiples, operational efficiency, and strategic capability. Takes 2-4 weeks to complete and surfaces the critical 80% of technical risks across AI/ML strategy, growth capacity, disaster recovery, cost optimization, product management, engineering practices, operations, team structure, and security. Used to: (1) Baseline current technology maturity, (2) Identify highest-impact improvement opportunities, (3) Prioritize investments based on ROI, (4) Establish metrics for measuring progress, (5) Support due diligence for acquisitions/exits.

Q: How long does a comprehensive technology assessment take?

A: Technology assessment timeline: Week 1: Kickoff and stakeholder interviews (CEO, CTO, engineering leads, operations)—understand business context, strategic goals, current challenges. Week 2-3: Technical deep dive (architecture review, code analysis, infrastructure assessment, vendor evaluation, team evaluation)—gather evidence across all assessment categories. Week 4: Analysis and report (synthesize findings, prioritize risks, develop recommendations, create roadmap)—deliver comprehensive report with prioritized action plan. Total: 4 weeks for comprehensive assessment. Quick assessment (high-level, focus on critical risks only): 1-2 weeks. Typical cost: $15K-$40K for comprehensive assessment; $5K-$15K for quick assessment.

Q: What are the most common technology risks uncovered in assessments?

A: Top 10 risks found in 80%+ of family office and portfolio company assessments: (1) “Big ball of mud” code—tangled dependencies, no clear architecture, high change risk. (2) Infrastructure debt—outdated servers, unsupported software, manual deployments. (3) Single points of knowledge—only one person understands critical systems. (4) Weak security posture—no MFA, unencrypted data, inadequate access controls. (5) No disaster recovery plan—business continuity at risk. (6) Poor testing coverage—changes cause regressions, quality issues. (7) Vendor lock-in—dependency on single vendor without alternatives. (8) Lack of monitoring—blind to system health, incidents discovered by users. (9) Technical roadmap absent—reactive decisions, no strategic technology plan. (10) Team skill gaps—critical capabilities missing in-house.

Q: How should family offices act on assessment findings?

A: Post-assessment action framework: Immediate (0-30 days): Address critical security risks (implement MFA, patch vulnerabilities, encrypt backups), Document disaster recovery procedures (even basic runbook better than nothing), Establish on-call rotation (ensure someone responsible for incidents). Short-term (1-6 months): Implement quick-win improvements (automated deployments, monitoring dashboards, testing for critical flows), Reduce highest technical debt (refactor most fragile code, upgrade end-of-life dependencies), Build team capability (training, documentation, knowledge transfer). Long-term (6-24 months): Execute strategic roadmap (architecture modernization, cloud migration, technology stack evolution), Institutionalize best practices (CI/CD, code review, incident postmortems), Build sustainable technology organization (hiring, retention, culture). Prioritize based on risk × impact; address security/compliance risks immediately regardless of cost.