refactor: redesign extractor services with performance optimizations#277
Merged
unhappychoice merged 14 commits intomainfrom Oct 5, 2025
Merged
refactor: redesign extractor services with performance optimizations#277unhappychoice merged 14 commits intomainfrom
unhappychoice merged 14 commits intomainfrom
Conversation
Split the monolithic extractor module into three distinct functional areas: - source_file_extractor/: Repository to source files extraction - SourceFileExtractor (formerly RepositoryExtractor) - source_code_parser/: File to code chunk parsing and analysis - SourceCodeParser (formerly CodeChunkExtractor) - CommonExtractor (formerly core/extractor) - LanguageRegistry, parsers/, ast_walker, etc. - challenge_generator/: Code chunk to typing challenge conversion - ChallengeGenerator (formerly ChallengeConverter) This refactoring improves code organization by: - Separating concerns into logical functional boundaries - Making dependencies and data flow more explicit - Reducing coupling between different extraction phases - Improving maintainability and testability All 626 tests pass and compilation succeeds. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Remove static methods from Language trait to make it dyn compatible - Create Languages struct in language.rs with all static methods - Delete src/domain/services/source_code_parser/language_registry.rs - Update all imports and usages from Language:: to Languages:: - Reorganize test structure to match new service architecture - Move language tests to models/language/language_tests.rs - Move extractor tests to source_code_parser/source_code_parser_tests.rs - Maintain same functionality with cleaner separation of concerns 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Extend infrastructure FileStorage with walk_directory method for file traversal - Remove duplicate FileStorage implementation from domain layer - Simplify SourceFileExtractor to use concrete infrastructure FileStorage - Add memory-based testing support with mock FileStorage implementation - Refactor file collection logic to use functional programming patterns - Update all references to use unified file storage infrastructure - Add proper error handling for non-existent paths in both real and mock implementations - Migrate tests to tests/unit directory with complete mock-based coverage 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Move DifficultyLevel from presentation layer to domain/models - Redesign ChallengeGenerator into focused components: - ChunkSplitter: handles code chunk splitting logic - CodeCharacterCounter: optimized character counting - ProgressTracker: tracks processing progress - Update all import paths for DifficultyLevel across codebase - Add comprehensive test coverage with snapshot testing - Add benchmark for performance monitoring - Include test fixtures for complex test cases 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Redesign chunk extraction architecture with focused modules: - ChunkExtractor: core extraction logic with parent comment optimization - CommentProcessor: comment range processing with coordinate transformation - CacheBuilder: efficient byte-to-char mapping cache - IndentProcessor: indentation processing logic - Optimize comment range calculation by reusing parent ranges (115ms → 17ms) - Add is_valid_comment_node() method to Language trait for all languages - Enhance extraction with proper coordinate transformation - Add comprehensive benchmarks for performance monitoring - Update integration and unit tests for new architecture 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Add #[allow(clippy::module_inception)] to modules that have the same name as their containing directory to suppress clippy warnings. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Replace unwrap_or_else(Vec::new) with unwrap_or_default() for better readability and performance. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Introduce type aliases SplitResult and add type complexity allow attribute to improve code readability and suppress clippy warnings. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Introduce ChunkExtractionContext to encapsulate extract_chunks parameters, reducing argument count from 8 to 1 and improving code maintainability. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Add Default trait implementations for SourceFileExtractor and FileStorage to follow Rust conventions and suppress clippy warnings. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Replace redundant closure with direct function reference for StepResult::ScannedFiles. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add type_complexity allow attributes for mock reporters - Mark unused variables with underscore prefix - Replace needless_borrow with direct value passing - Replace map_or with is_some_and for cleaner code - Add dead_code allow for test helper methods 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Apply automatic formatting to all Rust source files. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
WalkthroughReplaces the legacy extractor stack with new SourceFileExtractor, SourceCodeParser, and ChallengeGenerator services; adds DifficultyLevel and Languages domain APIs; implements per-language comment-node checks; introduces chunk-splitting, character counting, progress tracking, benchmarks, fixtures, and many test updates; removes old extractor/core/parser cache/visitor modules. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor User
participant LoadingScreen
participant SourceFileExtractor
participant SourceCodeParser
participant ChallengeGenerator
participant ProgressReporter as Progress
User->>LoadingScreen: Start extraction+generation
LoadingScreen->>SourceFileExtractor: collect_with_progress(repo_path, Progress)
SourceFileExtractor-->>LoadingScreen: Vec<PathBuf>
LoadingScreen->>SourceCodeParser: extract_chunks_with_progress(files+langs, options, Progress)
SourceCodeParser-->>LoadingScreen: Vec<CodeChunk>
LoadingScreen->>ChallengeGenerator: convert_with_progress(chunks, Progress)
ChallengeGenerator-->>LoadingScreen: Vec<Challenge>
LoadingScreen-->>User: Challenges ready
note over Progress: Reports Scanning → Parsing → Generating steps
sequenceDiagram
autonumber
participant Generator as ChallengeGenerator
participant Tracker as ProgressTracker
participant Pool as Rayon
participant Counter as CodeCharacterCounter
participant Splitter as ChunkSplitter
Generator->>Tracker: initialize(total_work)
Generator->>Pool: process chunks in parallel
Pool->>Counter: count_code_characters(chunk)
Pool-->>Generator: code_char_count
alt chunk exceeds difficulty limit
Generator->>Splitter: split(chunk, difficulty)
Splitter-->>Generator: truncated chunk(s)
end
Generator->>Tracker: increment_and_report()
Generator->>Tracker: finalize()
Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly related PRs
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (4)
🚧 Files skipped from review as they are similar to previous changes (2)
🧰 Additional context used🧬 Code graph analysis (2)src/infrastructure/storage/compressed_file_storage.rs (1)
src/domain/repositories/challenge_repository.rs (1)
🔇 Additional comments (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #277 +/- ##
==========================================
+ Coverage 22.57% 24.61% +2.04%
==========================================
Files 192 195 +3
Lines 21603 21720 +117
==========================================
+ Hits 4876 5347 +471
+ Misses 16727 16373 -354
🚀 New features to boost your workflow:
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 13
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/integration/languages/typing_core_common.rs (1)
107-116: Replace empty cache with a real byte-to-char map in tests
Passing&[]toextract_comment_rangesmeansbyte_to_char_cachedalways returns 0 (empty cache ⇒cache.last().unwrap_or(0)), so every comment range collapses to(0,0)— you need to build and pass a proper cache.
Intests/integration/languages/typing_core_common.rs:111, replaceCommentProcessor::extract_comment_ranges(&tree, code, language_obj.as_ref(), &[])with something like
let cache = CacheBuilder::build_byte_to_char_cache(code); CommentProcessor::extract_comment_ranges(&tree, code, language_obj.as_ref(), &cache)
🧹 Nitpick comments (11)
src/domain/models/languages/csharp.rs (1)
27-30: LGTM! Consider inlining the node kind check.The implementation correctly fulfills the
Languagetrait requirement foris_valid_comment_node.For a more concise implementation, you could inline the
node_kindvariable:- fn is_valid_comment_node(&self, node: tree_sitter::Node) -> bool { - let node_kind = node.kind(); - node_kind == "comment" - } + fn is_valid_comment_node(&self, node: tree_sitter::Node) -> bool { + node.kind() == "comment" + }src/domain/models/languages/swift.rs (1)
24-27: LGTM! Consider inlining the node kind check.The implementation correctly handles both single-line and multi-line comments for Swift, consistent with other language implementations.
For a more concise implementation:
- fn is_valid_comment_node(&self, node: tree_sitter::Node) -> bool { - let node_kind = node.kind(); - node_kind == "comment" || node_kind == "multiline_comment" - } + fn is_valid_comment_node(&self, node: tree_sitter::Node) -> bool { + matches!(node.kind(), "comment" | "multiline_comment") + }src/domain/models/languages/rust.rs (1)
27-30: LGTM! Consider usingmatches!for clarity.The implementation correctly handles both line and block comments for Rust, consistent with other language implementations like Java.
For improved readability:
- fn is_valid_comment_node(&self, node: tree_sitter::Node) -> bool { - let node_kind = node.kind(); - node_kind == "line_comment" || node_kind == "block_comment" - } + fn is_valid_comment_node(&self, node: tree_sitter::Node) -> bool { + matches!(node.kind(), "line_comment" | "block_comment") + }tests/integration/languages/extractor.rs (1)
19-19: Considerexpect()for clearer test failures.The migration from
CodeChunkExtractortoSourceCodeParseris correct. However,unwrap()in tests could benefit fromexpect()for more descriptive panic messages.Apply this diff for better error messages:
- let mut extractor = SourceCodeParser::new().unwrap(); + let mut extractor = SourceCodeParser::new().expect("Failed to create SourceCodeParser");Also applies to: 30-30
tests/unit/domain/models/language_tests.rs (1)
73-79: Consider additional edge case coverage forget_by_name.The basic tests are correct, but could benefit from testing edge cases like empty strings and case sensitivity (to ensure lowercase normalization works).
Consider adding these test cases:
#[test] fn language_registry_get_by_name_is_case_insensitive() { assert!(Languages::get_by_name("RUST").is_some()); assert!(Languages::get_by_name("Rust").is_some()); assert!(Languages::get_by_name("RuSt").is_some()); } #[test] fn language_registry_get_by_name_handles_edge_cases() { assert!(Languages::get_by_name("").is_none()); assert!(Languages::get_by_name(" ").is_none()); }benches/challenge_generator_bench.rs (1)
11-21: Consider removing redundantnew()method.Since
BenchProgressReporterderivesDefault, the explicitnew()method (lines 18-20) that just callsSelf::default()is redundant. Callers could useBenchProgressReporter::default()directly.Apply this diff if you want to simplify:
-impl BenchProgressReporter { - fn new() -> Self { - Self::default() - } -} - impl ProgressReporter for BenchProgressReporter {Or keep
new()if you prefer the explicitness for benchmark code readability.src/presentation/game/stage_repository.rs (1)
79-79: Use the importedHashMapfor consistency.Lines 79 and 91 still use
std::collections::HashMap::new()despite importingHashMapat line 9. For consistency with the refactoring applied at lines 48 and 66, use the imported alias.Apply this diff:
- difficulty_indices: std::collections::HashMap::new(), + difficulty_indices: HashMap::new(),- difficulty_indices: std::collections::HashMap::new(), + difficulty_indices: HashMap::new(),Also applies to: 91-91
src/domain/models/challenge.rs (1)
67-68: Moveuuidimport to the top of the file.The
use uuid;statements insidefrom_chunk()andfrom_content_and_chunk()are unconventional. Move the import to the top of the file with other imports for better consistency and readability.Apply this diff at the top of the file:
use super::{git_repository::GitRepository, DifficultyLevel}; use std::borrow::Cow; use std::path::Path; +use uuid::Uuid;Then remove the inline imports:
) -> Option<Self> { // Early validation if chunk.content.trim().is_empty() { return None; } - use uuid; - - let id = uuid::Uuid::new_v4().to_string(); + let id = Uuid::new_v4().to_string();comment_ranges: &[(usize, usize)], difficulty: Option<DifficultyLevel>, ) -> Self { - use uuid; - - let id = uuid::Uuid::new_v4().to_string(); + let id = Uuid::new_v4().to_string();Also applies to: 96-97
tests/unit/domain/services/challenge_generator/challenge_generator_tests.rs (1)
79-82: Useunwrap_or_default()for consistency with PR objectives.Line 81 uses
unwrap_or_else(|_| Vec::new()), but the PR objectives state thatunwrap_or_else(Vec::new)should be replaced withunwrap_or_default()for improved code quality.Apply this diff:
parser .extract_chunks_with_progress(files_to_process, &options, &progress) - .unwrap_or_else(|_| Vec::new()) + .unwrap_or_default()tests/integration/comment_processing_tests.rs (1)
188-189: Remove unused_convertervariable.The
ChallengeGeneratorinstance is created but never used, asChallenge::from_chunkis a static method that doesn't require it.Apply this diff:
- let _converter = ChallengeGenerator::new(); let challenge = Challenge::from_chunk(chunk, None).unwrap();tests/unit/domain/services/source_code_parser/source_code_parser_tests.rs (1)
180-193: Remove unused_convertervariable.Similar to the integration test, the
ChallengeGeneratorinstance is unused sinceChallenge::from_chunkis a static method.Apply this diff:
- let _converter = ChallengeGenerator::new(); let chunk = CodeChunk {
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (3)
Cargo.lockis excluded by!**/*.locktests/unit/domain/services/challenge_generator/snapshots/r#mod__unit__domain__services__challenge_generator__challenge_generator_tests__complex_commented_rust_challenges.snapis excluded by!**/*.snaptests/unit/domain/services/challenge_generator/snapshots/r#mod__unit__domain__services__challenge_generator__challenge_generator_tests__complex_rust_service_challenges.snapis excluded by!**/*.snap
📒 Files selected for processing (91)
Cargo.toml(1 hunks)benches/challenge_generator_bench.rs(1 hunks)benches/source_code_parser_bench.rs(1 hunks)src/domain/models/challenge.rs(2 hunks)src/domain/models/difficulty_level.rs(1 hunks)src/domain/models/extraction_options.rs(3 hunks)src/domain/models/language.rs(2 hunks)src/domain/models/languages/c.rs(1 hunks)src/domain/models/languages/cpp.rs(1 hunks)src/domain/models/languages/csharp.rs(1 hunks)src/domain/models/languages/dart.rs(1 hunks)src/domain/models/languages/go.rs(1 hunks)src/domain/models/languages/haskell.rs(1 hunks)src/domain/models/languages/java.rs(1 hunks)src/domain/models/languages/javascript.rs(1 hunks)src/domain/models/languages/kotlin.rs(1 hunks)src/domain/models/languages/php.rs(1 hunks)src/domain/models/languages/python.rs(1 hunks)src/domain/models/languages/ruby.rs(1 hunks)src/domain/models/languages/rust.rs(1 hunks)src/domain/models/languages/scala.rs(1 hunks)src/domain/models/languages/swift.rs(1 hunks)src/domain/models/languages/typescript.rs(1 hunks)src/domain/models/mod.rs(2 hunks)src/domain/repositories/challenge_repository.rs(1 hunks)src/domain/repositories/version_repository.rs(1 hunks)src/domain/services/challenge_generator/challenge_generator.rs(1 hunks)src/domain/services/challenge_generator/chunk_splitter.rs(1 hunks)src/domain/services/challenge_generator/code_character_counter.rs(1 hunks)src/domain/services/challenge_generator/mod.rs(1 hunks)src/domain/services/challenge_generator/progress_tracker.rs(1 hunks)src/domain/services/config_manager.rs(2 hunks)src/domain/services/extractor/challenge_converter.rs(0 hunks)src/domain/services/extractor/code_chunk_extractor.rs(0 hunks)src/domain/services/extractor/core/cache.rs(0 hunks)src/domain/services/extractor/core/extractor.rs(0 hunks)src/domain/services/extractor/core/mod.rs(0 hunks)src/domain/services/extractor/core/visitor.rs(0 hunks)src/domain/services/extractor/language_registry.rs(0 hunks)src/domain/services/extractor/mod.rs(0 hunks)src/domain/services/extractor/repository_extractor.rs(0 hunks)src/domain/services/history.rs(0 hunks)src/domain/services/mod.rs(1 hunks)src/domain/services/source_code_parser/cache_builder.rs(1 hunks)src/domain/services/source_code_parser/chunk_extractor.rs(1 hunks)src/domain/services/source_code_parser/comment_processor.rs(1 hunks)src/domain/services/source_code_parser/indent_processor.rs(1 hunks)src/domain/services/source_code_parser/mod.rs(1 hunks)src/domain/services/source_code_parser/source_code_parser.rs(1 hunks)src/domain/services/source_file_extractor/mod.rs(1 hunks)src/domain/services/source_file_extractor/source_file_extractor.rs(1 hunks)src/infrastructure/storage/file_storage.rs(4 hunks)src/presentation/cli/commands/game.rs(2 hunks)src/presentation/cli/views/repo_list_view.rs(2 hunks)src/presentation/cli/views/repo_play_view.rs(2 hunks)src/presentation/game/mod.rs(1 hunks)src/presentation/game/models/loading_steps/extracting_step.rs(2 hunks)src/presentation/game/models/loading_steps/finalizing_step.rs(1 hunks)src/presentation/game/models/loading_steps/generating_step.rs(2 hunks)src/presentation/game/models/loading_steps/mod.rs(0 hunks)src/presentation/game/models/loading_steps/scanning_step.rs(2 hunks)src/presentation/game/screens/loading_screen.rs(0 hunks)src/presentation/game/screens/title_screen.rs(1 hunks)src/presentation/game/session_manager.rs(1 hunks)src/presentation/game/stage_repository.rs(3 hunks)src/presentation/game/views/analytics/languages_view.rs(3 hunks)src/presentation/game/views/analytics/overview_view.rs(2 hunks)src/presentation/game/views/title/difficulty_selection_view.rs(1 hunks)src/presentation/game/views/typing/typing_header_view.rs(2 hunks)tests/fixtures/complex_commented_rust.rs(1 hunks)tests/fixtures/complex_rust_service.rs(1 hunks)tests/integration/comment_processing_tests.rs(4 hunks)tests/integration/indent_treesitter_tests.rs(7 hunks)tests/integration/languages/extractor.rs(2 hunks)tests/integration/languages/typescript/tsx_jsx.rs(4 hunks)tests/integration/languages/typing_core_common.rs(2 hunks)tests/integration/mod.rs(7 hunks)tests/unit/domain/models/challenge_tests.rs(1 hunks)tests/unit/domain/models/language_tests.rs(1 hunks)tests/unit/domain/models/mod.rs(1 hunks)tests/unit/domain/services/challenge_generator/challenge_generator_tests.rs(1 hunks)tests/unit/domain/services/challenge_generator/mod.rs(1 hunks)tests/unit/domain/services/extractor/core/mod.rs(0 hunks)tests/unit/domain/services/extractor/language_registry_tests.rs(0 hunks)tests/unit/domain/services/extractor/mod.rs(0 hunks)tests/unit/domain/services/history_tests.rs(0 hunks)tests/unit/domain/services/mod.rs(1 hunks)tests/unit/domain/services/source_code_parser/mod.rs(1 hunks)tests/unit/domain/services/source_code_parser/source_code_parser_tests.rs(9 hunks)tests/unit/domain/services/source_file_extractor/mod.rs(1 hunks)tests/unit/presentation/game/game_unit_tests.rs(1 hunks)
💤 Files with no reviewable changes (16)
- tests/unit/domain/services/extractor/language_registry_tests.rs
- src/presentation/game/models/loading_steps/mod.rs
- src/domain/services/history.rs
- src/domain/services/extractor/core/extractor.rs
- src/domain/services/extractor/core/cache.rs
- src/presentation/game/screens/loading_screen.rs
- src/domain/services/extractor/mod.rs
- src/domain/services/extractor/language_registry.rs
- src/domain/services/extractor/core/visitor.rs
- src/domain/services/extractor/challenge_converter.rs
- tests/unit/domain/services/history_tests.rs
- src/domain/services/extractor/core/mod.rs
- src/domain/services/extractor/repository_extractor.rs
- tests/unit/domain/services/extractor/mod.rs
- tests/unit/domain/services/extractor/core/mod.rs
- src/domain/services/extractor/code_chunk_extractor.rs
🧰 Additional context used
🧬 Code graph analysis (51)
src/domain/services/source_code_parser/cache_builder.rs (1)
src/domain/services/source_code_parser/comment_processor.rs (1)
byte_to_char_cached(95-101)
src/domain/models/languages/javascript.rs (2)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)src/domain/models/languages/typescript.rs (1)
is_valid_comment_node(27-30)
tests/unit/domain/models/language_tests.rs (2)
src/domain/models/language.rs (6)
get_color(133-140)get_display_name(142-149)get_by_name(126-131)name(11-11)display_name(14-16)color(40-42)src/presentation/ui/colors.rs (5)
lang_rust(108-110)lang_python(111-113)lang_javascript(114-116)lang_typescript(117-119)lang_default(156-158)
src/domain/models/languages/haskell.rs (1)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)
src/domain/models/languages/kotlin.rs (4)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)src/domain/models/languages/java.rs (1)
is_valid_comment_node(24-27)src/domain/models/languages/rust.rs (1)
is_valid_comment_node(27-30)src/domain/models/languages/swift.rs (1)
is_valid_comment_node(24-27)
src/domain/repositories/version_repository.rs (4)
src/domain/services/source_file_extractor/source_file_extractor.rs (1)
new(19-23)src/infrastructure/storage/file_storage.rs (2)
new(24-26)new(131-133)src/domain/services/config_manager.rs (1)
new(11-20)src/infrastructure/http/github_api_client.rs (2)
new(21-31)new(62-64)
src/presentation/game/views/analytics/overview_view.rs (1)
src/domain/models/language.rs (1)
get_display_name(142-149)
benches/challenge_generator_bench.rs (3)
tests/unit/domain/services/challenge_generator/challenge_generator_tests.rs (4)
new(22-24)set_step(42-44)set_current_file(46-48)set_file_counts(50-61)tests/unit/domain/services/source_file_extractor/mod.rs (4)
new(13-17)set_step(25-27)set_current_file(29-31)set_file_counts(33-44)src/domain/models/language.rs (1)
from_extension(111-115)
src/domain/models/languages/cpp.rs (4)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)src/domain/models/languages/dart.rs (1)
is_valid_comment_node(24-27)src/domain/models/languages/java.rs (1)
is_valid_comment_node(24-27)src/domain/models/languages/rust.rs (1)
is_valid_comment_node(27-30)
src/domain/models/languages/python.rs (1)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)
src/domain/models/languages/ruby.rs (16)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)src/domain/models/languages/c.rs (1)
is_valid_comment_node(24-27)src/domain/models/languages/cpp.rs (1)
is_valid_comment_node(27-30)src/domain/models/languages/csharp.rs (1)
is_valid_comment_node(27-30)src/domain/models/languages/dart.rs (1)
is_valid_comment_node(24-27)src/domain/models/languages/go.rs (1)
is_valid_comment_node(24-27)src/domain/models/languages/haskell.rs (1)
is_valid_comment_node(27-30)src/domain/models/languages/java.rs (1)
is_valid_comment_node(24-27)src/domain/models/languages/javascript.rs (1)
is_valid_comment_node(27-30)src/domain/models/languages/kotlin.rs (1)
is_valid_comment_node(27-30)src/domain/models/languages/php.rs (1)
is_valid_comment_node(24-27)src/domain/models/languages/python.rs (1)
is_valid_comment_node(27-30)src/domain/models/languages/rust.rs (1)
is_valid_comment_node(27-30)src/domain/models/languages/scala.rs (1)
is_valid_comment_node(27-30)src/domain/models/languages/swift.rs (1)
is_valid_comment_node(24-27)src/domain/models/languages/typescript.rs (1)
is_valid_comment_node(27-30)
src/presentation/game/views/analytics/languages_view.rs (1)
src/domain/models/language.rs (2)
get_display_name(142-149)display_name(14-16)
src/presentation/game/views/typing/typing_header_view.rs (1)
src/domain/models/language.rs (3)
get_color(133-140)display_name(14-16)get_display_name(142-149)
src/domain/services/source_code_parser/source_code_parser.rs (3)
src/domain/services/source_code_parser/parsers/mod.rs (2)
parse_with_thread_local(179-197)new(52-87)src/domain/services/source_code_parser/chunk_extractor.rs (1)
extract_chunks_from_tree(31-146)src/infrastructure/git/local/local_git_repository_client.rs (1)
get_repository_root(15-29)
src/domain/services/source_code_parser/comment_processor.rs (2)
src/domain/services/source_code_parser/parsers/mod.rs (1)
get_parser_registry(170-172)src/domain/services/source_code_parser/cache_builder.rs (1)
byte_to_char_cached(39-45)
src/domain/models/languages/php.rs (3)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)src/domain/models/languages/dart.rs (1)
is_valid_comment_node(24-27)src/domain/models/languages/java.rs (1)
is_valid_comment_node(24-27)
src/infrastructure/storage/file_storage.rs (3)
src/domain/services/source_file_extractor/source_file_extractor.rs (2)
new(19-23)default(13-15)src/domain/repositories/version_repository.rs (1)
new(16-21)src/domain/services/config_manager.rs (1)
new(11-20)
src/domain/models/languages/swift.rs (4)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)src/domain/models/languages/java.rs (1)
is_valid_comment_node(24-27)src/domain/models/languages/kotlin.rs (1)
is_valid_comment_node(27-30)src/domain/models/languages/rust.rs (1)
is_valid_comment_node(27-30)
src/domain/models/languages/java.rs (3)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)src/domain/models/languages/kotlin.rs (1)
is_valid_comment_node(27-30)src/domain/models/languages/rust.rs (1)
is_valid_comment_node(27-30)
src/domain/models/languages/go.rs (2)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)src/domain/models/languages/c.rs (1)
is_valid_comment_node(24-27)
benches/source_code_parser_bench.rs (2)
src/domain/services/source_code_parser/parsers/mod.rs (1)
parse_with_thread_local(179-197)src/domain/services/source_code_parser/chunk_extractor.rs (1)
extract_chunks_from_tree(31-146)
src/domain/services/challenge_generator/challenge_generator.rs (5)
src/domain/services/challenge_generator/code_character_counter.rs (2)
default(58-60)new(7-9)src/domain/services/challenge_generator/chunk_splitter.rs (2)
default(186-188)new(14-18)src/domain/models/challenge.rs (3)
new(18-29)from_chunk(58-86)from_content_and_chunk(88-113)src/domain/services/challenge_generator/progress_tracker.rs (1)
new(16-22)src/domain/models/difficulty_level.rs (1)
applicable_difficulties(44-66)
src/presentation/cli/views/repo_list_view.rs (1)
src/domain/models/language.rs (2)
get_display_name(142-149)get_color(133-140)
src/presentation/cli/commands/game.rs (1)
src/domain/models/language.rs (2)
validate_languages(96-109)get_supported_languages(85-94)
src/domain/services/challenge_generator/chunk_splitter.rs (2)
src/domain/services/challenge_generator/code_character_counter.rs (2)
new(7-9)default(58-60)src/domain/services/challenge_generator/challenge_generator.rs (2)
new(22-27)default(16-18)
tests/unit/domain/services/source_file_extractor/mod.rs (3)
src/domain/services/source_file_extractor/source_file_extractor.rs (2)
new(19-23)with_storage(25-27)src/infrastructure/storage/file_storage.rs (2)
new(24-26)new(131-133)src/presentation/game/screens/loading_screen.rs (10)
new(85-112)set_step(19-19)set_step(36-36)set_step(302-306)set_current_file(20-20)set_current_file(37-37)set_current_file(308-310)set_file_counts(21-27)set_file_counts(38-45)set_file_counts(312-343)
tests/integration/mod.rs (3)
src/domain/services/source_code_parser/parsers/mod.rs (1)
parse_with_thread_local(179-197)src/domain/models/language.rs (1)
from_extension(111-115)src/domain/services/source_code_parser/chunk_extractor.rs (1)
extract_chunks_from_tree(31-146)
src/domain/models/languages/dart.rs (4)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)src/domain/models/languages/java.rs (1)
is_valid_comment_node(24-27)src/domain/models/languages/kotlin.rs (1)
is_valid_comment_node(27-30)src/domain/models/languages/php.rs (1)
is_valid_comment_node(24-27)
src/domain/services/source_code_parser/chunk_extractor.rs (4)
src/domain/services/source_code_parser/parsers/mod.rs (1)
get_parser_registry(170-172)src/domain/services/source_code_parser/cache_builder.rs (3)
build_line_cache(27-37)build_byte_to_char_cache(4-25)byte_to_char_cached(39-45)src/domain/services/source_code_parser/comment_processor.rs (3)
extract_comment_ranges(11-30)convert_parent_comment_ranges_to_chunk(54-93)byte_to_char_cached(95-101)src/domain/services/source_code_parser/indent_processor.rs (1)
extract_and_normalize_indentation(6-31)
src/domain/models/languages/csharp.rs (1)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)
src/domain/services/challenge_generator/code_character_counter.rs (2)
src/domain/services/challenge_generator/chunk_splitter.rs (1)
new(14-18)src/domain/services/challenge_generator/challenge_generator.rs (1)
new(22-27)
src/domain/models/extraction_options.rs (1)
src/domain/models/language.rs (2)
all_file_patterns(78-83)all_languages(57-76)
tests/integration/comment_processing_tests.rs (4)
src/domain/services/source_code_parser/indent_processor.rs (1)
extract_and_normalize_indentation(6-31)src/domain/services/source_code_parser/parsers/mod.rs (1)
parse_with_thread_local(179-197)src/domain/services/source_code_parser/comment_processor.rs (1)
extract_comment_ranges(11-30)src/domain/models/challenge.rs (1)
from_chunk(58-86)
src/domain/services/config_manager.rs (3)
src/domain/services/source_file_extractor/source_file_extractor.rs (1)
new(19-23)src/infrastructure/storage/file_storage.rs (2)
new(24-26)new(131-133)src/domain/repositories/version_repository.rs (1)
new(16-21)
src/domain/models/languages/scala.rs (3)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)src/domain/models/languages/java.rs (1)
is_valid_comment_node(24-27)src/domain/models/languages/rust.rs (1)
is_valid_comment_node(27-30)
tests/integration/languages/typing_core_common.rs (3)
src/domain/services/source_code_parser/parsers/mod.rs (1)
parse_with_thread_local(179-197)src/domain/models/language.rs (1)
get_by_name(126-131)src/domain/services/source_code_parser/comment_processor.rs (1)
extract_comment_ranges(11-30)
src/presentation/cli/views/repo_play_view.rs (1)
src/domain/models/language.rs (2)
get_display_name(142-149)get_color(133-140)
src/presentation/game/models/loading_steps/scanning_step.rs (2)
src/domain/services/source_file_extractor/source_file_extractor.rs (1)
new(19-23)src/infrastructure/storage/file_storage.rs (2)
new(24-26)new(131-133)
src/presentation/game/models/loading_steps/extracting_step.rs (3)
src/domain/services/source_code_parser/source_code_parser.rs (1)
new(20-22)src/domain/services/source_code_parser/parsers/mod.rs (1)
new(52-87)src/domain/models/language.rs (1)
from_extension(111-115)
src/domain/models/languages/rust.rs (2)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)src/domain/models/languages/java.rs (1)
is_valid_comment_node(24-27)
src/domain/models/languages/typescript.rs (2)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)src/domain/models/languages/javascript.rs (1)
is_valid_comment_node(27-30)
tests/integration/indent_treesitter_tests.rs (5)
src/domain/services/source_code_parser/parsers/mod.rs (2)
parse_with_thread_local(179-197)get_parser_registry(170-172)src/domain/services/source_code_parser/comment_processor.rs (1)
extract_comment_ranges(11-30)src/domain/services/source_code_parser/chunk_extractor.rs (1)
extract_chunks_from_tree(31-146)src/domain/models/challenge.rs (1)
from_chunk(58-86)src/domain/services/source_code_parser/indent_processor.rs (1)
extract_and_normalize_indentation(6-31)
src/domain/services/source_file_extractor/source_file_extractor.rs (3)
src/infrastructure/storage/file_storage.rs (3)
default(125-127)new(24-26)new(131-133)src/domain/models/extraction_options.rs (1)
default(13-100)src/domain/models/language.rs (1)
from_extension(111-115)
src/presentation/game/models/loading_steps/generating_step.rs (2)
src/domain/services/challenge_generator/challenge_generator.rs (1)
new(22-27)src/domain/repositories/challenge_repository.rs (1)
new(35-44)
tests/integration/languages/extractor.rs (2)
src/domain/services/source_code_parser/source_code_parser.rs (1)
new(20-22)src/domain/services/source_code_parser/parsers/mod.rs (1)
new(52-87)
src/domain/models/language.rs (16)
src/domain/models/languages/c.rs (2)
is_valid_comment_node(24-27)name(9-11)src/domain/models/languages/cpp.rs (2)
is_valid_comment_node(27-30)name(9-11)src/domain/models/languages/csharp.rs (2)
is_valid_comment_node(27-30)name(9-11)src/domain/models/languages/dart.rs (2)
is_valid_comment_node(24-27)name(9-11)src/domain/models/languages/go.rs (2)
is_valid_comment_node(24-27)name(9-11)src/domain/models/languages/haskell.rs (2)
is_valid_comment_node(27-30)name(9-11)src/domain/models/languages/java.rs (2)
is_valid_comment_node(24-27)name(9-11)src/domain/models/languages/javascript.rs (2)
is_valid_comment_node(27-30)name(9-11)src/domain/models/languages/kotlin.rs (2)
is_valid_comment_node(27-30)name(9-11)src/domain/models/languages/php.rs (2)
is_valid_comment_node(24-27)name(9-11)src/domain/models/languages/python.rs (2)
is_valid_comment_node(27-30)name(9-11)src/domain/models/languages/ruby.rs (2)
is_valid_comment_node(27-30)name(9-11)src/domain/models/languages/rust.rs (2)
is_valid_comment_node(27-30)name(9-11)src/domain/models/languages/scala.rs (2)
is_valid_comment_node(27-30)name(9-11)src/domain/models/languages/swift.rs (2)
is_valid_comment_node(24-27)name(9-11)src/domain/models/languages/typescript.rs (2)
is_valid_comment_node(27-30)name(9-11)
src/domain/models/languages/c.rs (2)
src/domain/models/language.rs (1)
is_valid_comment_node(45-45)src/domain/models/languages/java.rs (1)
is_valid_comment_node(24-27)
tests/unit/domain/services/challenge_generator/challenge_generator_tests.rs (4)
benches/challenge_generator_bench.rs (4)
new(18-20)set_step(24-24)set_current_file(26-26)set_file_counts(28-39)src/domain/services/challenge_generator/challenge_generator.rs (2)
new(22-27)default(16-18)src/domain/models/challenge.rs (2)
new(18-29)from_chunk(58-86)src/domain/models/language.rs (1)
from_extension(111-115)
tests/integration/languages/typescript/tsx_jsx.rs (2)
src/domain/services/source_code_parser/source_code_parser.rs (1)
new(20-22)src/domain/services/source_code_parser/parsers/mod.rs (1)
new(52-87)
src/presentation/game/stage_repository.rs (3)
src/presentation/game/screens/title_screen.rs (1)
new(45-53)src/domain/models/challenge.rs (1)
new(18-29)src/presentation/game/session_manager.rs (1)
new(82-93)
tests/unit/domain/services/source_code_parser/source_code_parser_tests.rs (5)
src/domain/models/language.rs (4)
from_extension(111-115)name(11-11)validate_languages(96-109)detect_from_path(117-124)src/domain/services/source_file_extractor/source_file_extractor.rs (1)
new(19-23)src/domain/services/challenge_generator/challenge_generator.rs (1)
new(22-27)src/domain/services/source_code_parser/source_code_parser.rs (1)
new(20-22)src/domain/models/challenge.rs (1)
from_chunk(58-86)
unhappychoice
changed the title
- Replace "multiline_comment" with "block_comment" in Kotlin parser - Add new() methods to CompressedFileStorage and FileStorage - Implement Default trait for FileStorage - Update snapshot test for Kotlin comprehensive extraction 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Owner
Author
|
Ref: #267 |
5 tasks
22 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR is a comprehensive refactoring of the extractor services layer, focusing on improving architecture, performance, and code quality.
Major Changes
Architecture Refactoring
Performance Optimizations
Test Plan
cargo test)cargo clippy --all-targets --all-features -- -D warnings)cargo fmt🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Refactor
Performance
Tests/Chores