Word Tokenizer and Vocabulary

Language models work with tokens, not raw text. A simple tokenizer lowercases the text and treats every run of letters a-z and digits 0-9 as a word; everything else (spaces, punctuation, other characters) separates words.

Build the vocabulary: the distinct words, sorted by how often they appear (most frequent first), with ties broken alphabetically. A word's position in this list is its token ID.

Implement build_vocabulary(text).

Input

text: Any text.

Output

Return the vocabulary as a list of words. Return an empty list if the text has no words.

Constraints

0 ≤ length of text ≤ 100,000

Examples

Example 1

Input:
text = "The cat sat on the mat. The end!"

Expected output:
["the", "cat", "end", "mat", "on", "sat"]

Explanation

Lowercased, the words are the, cat, sat, on, the, mat, the, end. "the" appears 3 times; the rest appear once and are sorted alphabetically: ["the", "cat", "end", "mat", "on", "sat"].

Example 2

Input:
text = "AI, ai and A.I. (2024)"

Expected output:
["ai", "2024", "a", "and", "i"]

Explanation

Words: ai, ai, and, a, i, 2024. "ai" appears twice; the others once, sorted alphabetically with digits before letters: ["ai", "2024", "a", "and", "i"].

References

Run the sample tests to check your code without submitting.

Example 1 Not run

Input:

text = "The cat sat on the mat. The end!"

Expected output:

["the", "cat", "end", "mat", "on", "sat"]

Your output

Example 2 Not run

Input:

text = "AI, ai and A.I. (2024)"

Expected output:

["ai", "2024", "a", "and", "i"]

Your output

4 hidden tests also run when you submit. If your code fails one, you'll see that test. Solve the challenge to see them all.

Online Rust Compiler

Write, compile, run, and debug Rust code online in a full-featured browser IDE. Work with multiple files, save your workspace, and share or collaborate in real time.

Version

Pre-installed Libs

🚀 252,702 total executions (135 this month)

Output

Practice Rust with coding challenges

Solve hands-on problems right in this editor: run the sample tests, submit, and get an AI code review.

All Rust challenges

Easy

Rectangle Perimeter

Compute the perimeter of a rectangle from its sides.

You'll learn ArithmeticReturn values

BeginnerLearning-pathVariables

View solutions (12)

Easy

Distance Between Points

Compute the straight-line distance between two points.

You'll learn Pythagorean theoremSquare roots

BeginnerFunctionsLearning-pathmath

View solutions (7)

Easy

Days in a Month

Return how many days a month has, including leap years.

You'll learn Calendar rulesHelper functions

BeginnerConditionsFunctionsLearning-path

View solutions (6)

Easy

Average of Three

Average three whole numbers.

You'll learn ArithmeticDivision

BeginnerLearning-pathVariables

View solutions (6)

Easy

Dot Product of Two Vectors

Multiply two vectors element by element and add the results: the core operation of every neuron.

You'll learn VectorsDot product

Neural networksaimachine learning

View solutions (4)

Easy

Largest of Three

Pick the largest of three numbers without a built-in max.

You'll learn ComparisonsCombining conditions

BeginnerConditionsLearning-path

View solutions (4)

Easy

Prime Number Check

Decide whether a number is prime.

You'll learn DivisibilityLoop optimization

BeginnerLoopsNumbersStudent-beginner

View solutions (3)

Easy

Number Triangle

Build a triangle of increasing numbers.

You'll learn Growing rowsCounters across rows

BeginnerLearning-pathNested-loopsPatterns

View solutions (3)

Easy

Armstrong Number

Check whether a number equals the sum of its digits raised to the digit count.

You'll learn Digit extractionPowers

BeginnerNumbersStudent-beginner

View solutions (3)

Easy

Row Sums

Add up every row of a grid.

You'll learn 2D listsNested loops

BeginnerGridsLearning-path

View solutions (3)

Easy

Min and Max Together

Find the smallest and largest value in one pass.

You'll learn Single passMultiple return values

ArraysBeginnerFunctionsLearning-path

View solutions (3)

Easy

Count Pairs With a Sum

Count pairs of positions whose values add up to a target.

You'll learn Pairs of indexesAvoiding double counts

ArraysBeginnerLearning-pathNested-loops

View solutions (3)

🦀 About This Rust Online Executor

The CodeUtility Rust Executor lets you write and run Rust code directly in your browser - no installation, toolchain setup, or local environment required. It’s powered by a secure sandbox that supports real Rust compiler versions 1.70, 1.72, 1.73, and 1.74.

This tool compiles and executes your Rust programs in the cloud using a genuine rustc compiler, ensuring accurate and consistent results. You can experiment with variables, ownership, borrowing, pattern matching, traits, generics, and other modern Rust features with ease.

It’s ideal for learning Rust’s core concepts - especially memory safety, error handling, and concurrency - without needing to install Cargo or any local setup. Everything runs directly in your browser within seconds.

More than a code runner

Use a multi-file editor, an isolated execution terminal, runtime-specific libraries, persistent workspaces, sharing, and real-time collaboration from the same executor.

How to use this executor?

Choose a runtime, open or create a file, write your code, and select Run. Output appears in the terminal, while signed-in users can keep files in a workspace, install libraries, and collaborate through shares.

Select a version and edit the current file or open another workspace file.
Run the code, provide input in the terminal when requested, and review saved run history.
Use libraries, workspace tools, sharing, and chat when your task needs more than one snippet.

Start coding

VersionSelect a runtime before running. Installed libraries belong to the selected language version.
Open files lists unopened text files. Select one to open it in an available editor tab.
Run executes the active file with its workspace available to the program.
Stop ends the current execution before its time limit.
Advanced Mode opens the IDE layout with a file explorer and movable, resizable terminal.

Workspace and libraries

New file creates a file in the selected parent directory and opens it in the editor.
New folder creates a directory, up to the supported nesting depth.
Upload files adds one or more safe files to a directory.
Download saves a file or a zipped directory to your device.
Copy duplicates a workspace file or directory.
Rename changes a file name and updates its open tab and history.
Move relocates an item; files and directories can also be dragged.
Delete removes an item after confirmation and closes its open editor tab.
Pre-installed Libs shows packages already included in this runtime.
Add Lib installs a private package for your account and selected runtime.

Terminal and assistance

Run history opens previous and next saved outputs.
Copy output copies the current terminal text.
Clear output removes all saved run outputs after confirmation.
Explain summarizes the active code.
Recents opens recent explanations and results.
Settings changes fonts, themes, and visible toolbar buttons.

Sharing and collaboration

Share publishes the current file, a directory, or the whole workspace as a snapshot or live workspace. You control link access and who may save changes.
Logs records saved collaborator actions in a shared workspace.
Share chat lets signed-in participants message, mention people, share code blocks, and see online or typing members.
Live collaboration shows collaborator cursors and saved file updates. View-only users can load the latest version or keep temporary edits.

Quotas and limits

Execution time is limited to 20 seconds per server run. Use Stop when you want to end it sooner.
Workspace storage allows 100 files, 100 directories, and 10 MiB total for each signed-in user workspace and language.
Files and folders are limited to 1 MiB per file and 5 nested directory levels. Files generated by your code follow the same quota.
Private libraries are limited to 10 installed packages for each account and runtime.
Share snapshots can contain up to 100 files, 1 MiB per file, and 2 MiB total.
Share invitations support up to 25 people per share, with access and saving permissions managed separately.
Share chat allows 2,000 characters per message and 10,000 messages per share.
Quota enforcement also applies while code is running. A write that exceeds file count, size, storage, or nesting limits is rejected safely.

Good to know Only run code and install packages you trust. Execution is isolated and limited, but your program can still modify files inside a writable workspace up to its enforced quota.

💡 Rust Basics & Examples You Can Try Above

1. Declaring Variables and Constants

Use let to declare variables. Use const for constants and mut for mutable variables.

let x = 10;
let pi: f64 = 3.14;
let name = "Alice";
let is_active = true;

const MAX_USERS: u32 = 100;

2. Conditionals (if / match)

Rust uses if for conditions and match for pattern matching.

let x = 2;
if x == 1 {
    println!("One");
} else if x == 2 {
    println!("Two");
} else {
    println!("Other");
}

match x {
    1 => println!("One"),
    2 => println!("Two"),
    _ => println!("Other"),
}

3. Loops

Rust supports for, while, and infinite loop.

for i in 0..3 {
    println!("{}", i);
}

let mut n = 3;
while n > 0 {
    println!("{}", n);
    n -= 1;
}

4. Arrays

Fixed-size arrays in Rust use square brackets. Use slices for views.

let nums = [10, 20, 30];
println!("{}", nums[1]);

5. Vectors (Dynamic Arrays)

Vec is used for dynamic arrays.

let mut fruits = vec!["apple", "banana"];
fruits.push("cherry");
fruits.pop();

for fruit in &fruits {
    println!("{}", fruit);
}

6. Console Input/Output

Use println! for output and std::io for input.

use std::io;

let mut name = String::new();
println!("Enter your name:");
io::stdin().read_line(&mut name).unwrap();
println!("Hello, {}", name.trim());

7. Functions

Functions use fn and can return values using ->.

fn greet(name: &str) -> String {
    format!("Hello, {}", name)
}

println!("{}", greet("Alice"));

8. HashMaps

Use std::collections::HashMap for key-value data.

use std::collections::HashMap;

let mut person = HashMap::new();
person.insert("name", "Bob");
println!("{}", person["name"])

9. Error Handling

Rust handles errors using Result and match or unwrap().

fn fail() -> Result<(), String> {
    Err("Something went wrong".to_string())
}

match fail() {
    Ok(_) => println!("Success"),
    Err(e) => println!("{}", e),
}

10. File I/O

Use std::fs to read and write files.

use std::fs;

fs::write("file.txt", "Hello File").unwrap();
let content = fs::read_to_string("file.txt").unwrap();
println!("{}", content);

11. String Manipulation

Strings in Rust are heap-allocated and support many methods.

let mut text = String::from(" Hello Rust ");
println!("{}", text.trim());
println!("{}", text.to_uppercase());
text = text.replace("Hello", "Hi");
println!("{}", text);

12. Structs & Methods

Rust uses struct for data types and impl for methods.

struct Person {
    name: String,
}

impl Person {
    fn greet(&self) -> String {
        format!("Hi, I'm {}", self.name)
    }
}

let p = Person { name: "Alice".to_string() };
println!("{}", p.greet());

13. References & Borrowing

Rust uses & to borrow values and &mut for mutable references.

fn update(x: &mut i32) {
    *x += 1;
}

let mut value = 10;
update(&mut value);
println!("{}", value);