Table of Contents
Type: Full Masters Dissertation | Subject: Computer Science | Level: Masters | Word Count: ~14,000 words
This model Full Masters Dissertation was produced by an Essays UK specialist as reference material for learning purposes only. It is a worked example to illustrate structure, argument and academic style; it is not intended to be submitted as a student’s own work. For support in this field, see our computer science writing specialists.
Overview: This model dissertation presents a complete MSc Computer Science project on detecting phishing URLs using supervised machine learning. It moves from motivation and a critical literature review through requirements, system design, implementation and a rigorous empirical evaluation, and closes with a reflective discussion of findings, limitations and future work. All experimental figures are illustrative and are included to demonstrate how results should be reported and interpreted in a research dissertation.
Phishing remains one of the most persistent and economically damaging forms of social engineering, and the Uniform Resource Locator (URL) is frequently the first observable artefact of an attack. This dissertation investigates whether lightweight, URL-only features can support accurate, real-time classification of phishing links without recourse to page content, third-party reputation services or user interaction. A supervised learning framework is designed and implemented that extracts a set of lexical, host-based and derived features from raw URL strings and classifies them using a range of algorithms, from interpretable baselines to ensemble and neural models.
The framework was evaluated on an illustrative, balanced corpus of approximately 95,000 URLs drawn from publicly characterised phishing feeds and reputable benign sources. Using stratified cross-validation and a held-out test set, the strongest model, a gradient-boosted tree ensemble, achieved an illustrative test accuracy of 96.8 per cent, an F1 score of 0.967 and an area under the receiver operating characteristic curve (AUC) of 0.993. Feature-importance analysis indicated that URL length, the presence of a raw IP address in the host, the number of subdomains, the age of the registered domain and the misplacement of security-suggestive tokens were the most discriminative signals. The results support the central claim that URL-only classification can achieve high accuracy at low latency, making it suitable as a first line of defence within a layered anti-phishing architecture. The dissertation contributes a reproducible evaluation methodology, a critical appraisal of feature design and a candid discussion of the concept drift and adversarial evasion that constrain any static model.
Phishing is a form of social engineering in which an attacker impersonates a trusted entity in order to deceive a victim into disclosing credentials, financial information or other sensitive data, or into installing malicious software. Although the attack surface has widened to include voice calls, instant messaging and QR codes, the hyperlink embedded in an email or message remains the dominant delivery mechanism. The Uniform Resource Locator (URL) that a victim is invited to click is therefore one of the earliest and most reliably observable artefacts of a phishing campaign, often available for inspection before any page is rendered and before any personal data is entered. This early availability is precisely what makes the URL an attractive object of automated analysis: a classifier that can judge a link in milliseconds, using only the string itself, can intervene before a victim is exposed to a convincing counterfeit page.
The economic and operational motivation for improving phishing detection is substantial and well documented across the information-security literature and industry reporting (APWG, 2023). Phishing underpins a large proportion of data breaches and is a common precursor to ransomware deployment and business email compromise. Defences have historically relied on blacklists: centrally maintained registries of known-malicious URLs that browsers and mail gateways consult before allowing navigation. Blacklists are accurate when they match, impose negligible runtime cost and are easy to reason about. Their weakness is temporal. Phishing sites are frequently ephemeral, with many remaining live for only a few hours, and attackers routinely generate large numbers of distinct URLs through automated kits, algorithmically generated domains and URL-shortening services. A detection mechanism that depends on prior knowledge of a specific URL will always lag the attacker by the time required to discover, verify and distribute the new entry.
Machine learning offers a complementary paradigm. Rather than memorising individual malicious URLs, a learned model generalises from the statistical regularities that distinguish phishing links from legitimate ones. If those regularities can be captured by features extracted from the URL alone, detection becomes possible for previously unseen links, closing part of the window of exposure that blacklists leave open. The research reported in this dissertation sits firmly within this paradigm and asks a focused question: how far can URL-only machine learning go, and under what conditions does it break down?
Much of the published work on phishing detection relies on features that are either expensive to compute at scale or unavailable at the moment a decision is required. Content-based approaches that parse the target page’s HTML, inspect its forms or compare its visual appearance to known brands are powerful, but they require the page to be fetched, which introduces latency, consumes bandwidth and may itself expose infrastructure to the attacker. Reputation-based approaches that query WHOIS registries, certificate transparency logs or third-party scoring services introduce external dependencies, rate limits and privacy considerations. There is, consequently, a clear practical niche for a detector that uses nothing beyond the URL string and a small amount of locally cached information, trading a degree of accuracy for speed, independence and privacy.
The problem is complicated by two properties of the domain that any honest treatment must confront. The first is concept drift: the distribution of phishing URLs is not stationary. Attackers adapt, defensive signals that work today may be evaded tomorrow, and a model trained on historical data will degrade over time. The second is adversarial pressure: unlike many machine-learning problems, where the data-generating process is indifferent to the model, here an active adversary has both the motive and the means to craft inputs that defeat the classifier. A dissertation that reports a high accuracy figure without engaging with these properties would present a misleadingly optimistic picture. This project therefore treats robustness and the limits of generalisation as first-class concerns rather than afterthoughts.
The overall aim of this dissertation is to design, implement and critically evaluate a machine-learning framework that classifies URLs as phishing or legitimate using only features derivable from the URL string and lightweight local metadata, and to establish the conditions under which such a framework is and is not effective.
This aim is decomposed into the following objectives:
The investigation is guided by three research questions:
The scope of the project is deliberately bounded to keep the research question tractable within the time and resources available for a Masters dissertation. The framework classifies individual URLs in isolation; it does not model campaigns, sender reputation or user behaviour. It uses only features that can be computed from the URL string together with inexpensive, locally cacheable metadata; it does not fetch or render target pages. The evaluation uses a static, pre-collected corpus rather than a live feed, which means that the reported performance reflects a snapshot of the threat landscape rather than sustained operational accuracy. These boundaries are limitations as well as design choices, and the discussion chapter returns to them explicitly. The dataset figures and experimental results reported throughout are illustrative, generated to demonstrate sound reporting practice, and should not be cited as empirical facts about any specific operational system.
The remainder of the dissertation is organised as follows. Chapter 2 reviews the literature and establishes the research gap. Chapter 3 sets out the research design, requirements and methodology, including the dataset, feature set and evaluation strategy. Chapter 4 presents the system design and the rationale behind its modular architecture. Chapter 5 describes the implementation and the engineering decisions it entailed. Chapter 6 reports the empirical results and their analysis. Chapter 7 discusses the findings in relation to the research questions and the literature, and Chapter 8 concludes with a summary of contributions, limitations and avenues for future work.
This chapter surveys the body of work on automated phishing detection, with the dual purpose of situating the present study within the field and of exposing the gap it addresses. The review is organised thematically rather than chronologically. It begins by clarifying the phenomenon and its taxonomy, then examines the three broad families of detection technique: blacklisting, heuristic and rule-based methods, and machine learning. Within the machine-learning family it gives particular attention to feature engineering for URLs, to the algorithms that have been applied and to the increasingly prominent role of deep learning. It then considers the datasets and evaluation metrics that the community has adopted, because weaknesses in evaluation practice are themselves part of the research gap. The chapter closes by articulating that gap and the contribution the dissertation makes to closing it.
Phishing is best understood as a class of social-engineering attacks that exploit human trust rather than, or in addition to, technical vulnerabilities. The canonical form is credential harvesting, in which a victim is lured to a counterfeit login page that captures a username and password. Variants include spear phishing, which targets specific individuals using personalised pretexts; whaling, which targets senior executives; and clone phishing, which reproduces a legitimate message with malicious substitutions. From a detection standpoint the salient commonality is that almost all of these variants culminate in a request to visit a URL under the attacker’s control. This observation justifies the field’s sustained interest in URL analysis as a detection locus and underpins the scope decision taken in this project.
A useful distinction, drawn consistently across the surveyed literature, separates the signals available for analysis into three layers: the URL string itself, the content of the target page and the behaviour of the surrounding network and infrastructure (Sahoo, Liu and Hoi, 2017). Each layer offers a different trade-off between richness and cost. The URL layer is the cheapest and the earliest to become available; the content layer is richer but requires fetching the page; the infrastructure layer is richest of all but depends on external services and historical observation. The present study confines itself to the first layer, augmented only by lightweight, cacheable metadata, precisely because that layer is where low-latency intervention is feasible.
The earliest and still most widely deployed defence is the blacklist, a maintained registry of known-malicious URLs or domains that clients consult before permitting navigation. Blacklists are attractive for operational reasons: a match yields a confident verdict with negligible computation and no risk of a false positive beyond the accuracy of the list itself. Their limitation is coverage over time. Because phishing URLs are frequently short-lived and generated in bulk, a newly minted URL will not appear on any list until it has been observed, reported, verified and distributed, a pipeline that inevitably trails the attacker. Studies of blacklist freshness have repeatedly found that a substantial fraction of phishing URLs are never blacklisted during their active lifetime, or are blacklisted only after the period of greatest risk has passed. Whitelists, the mirror-image approach of permitting only known-good destinations, avoid false negatives but are impractically restrictive for general web use. The consensus in the literature is that reputation lists are a necessary but insufficient component of defence, motivating the predictive approaches that follow.
Between blacklisting and full machine learning lies a tradition of hand-crafted heuristics: expert-authored rules that flag suspicious properties of a URL or page. Typical rules target the presence of an IP address in place of a domain name, the use of security-suggestive words such as “secure” or “account” in unexpected positions, excessive URL length, the presence of the at-symbol, or a mismatch between the visible brand and the registered domain. Heuristic systems are transparent and require no training data, which makes them easy to audit and to explain to end users. Their weaknesses are brittleness and maintenance cost: each rule encodes a human hypothesis about attacker behaviour, and as attackers adapt, the rules must be rewritten by hand. Rule-based associative-classification work, such as that of Abdelhamid, Ayesh and Thabtah (2014), can be seen as a bridge between heuristics and learning, in that it mines rules automatically from data while retaining the interpretability of a rule set. The present study inherits the heuristic tradition’s feature insights but delegates the weighting of those features to learned models rather than to hand-tuned thresholds.
The dominant contemporary paradigm casts phishing detection as a supervised binary classification problem: given a labelled corpus of phishing and legitimate examples, learn a function that maps features to a class. The appeal is generalisation. A well-trained model captures the statistical regularities that separate the classes and can therefore classify URLs it has never encountered, addressing the central weakness of blacklists. The literature within this paradigm can be organised along two axes: the features used and the algorithms applied.
Feature engineering is widely regarded as the single most consequential design decision in URL-based detection. Three families recur throughout the literature. Lexical features are computed directly from the URL string and include its length, the counts of particular characters such as dots, hyphens, digits and the at-symbol, the number of subdomains, the length and composition of the path and query, and the presence of suspicious tokens. Host-based features describe the host named in the URL and include whether it is a raw IP address, the top-level domain, the length of the registered domain and, where cheaply available, the age of the registration and whether a shortening service is involved. Derived or statistical features capture higher-order properties such as the ratio of digits to letters, the entropy of the host or path, and the presence of brand names in positions that suggest impersonation. Sahingoz and colleagues (2019) demonstrated that purely URL-derived natural-language features can support high accuracy without any access to page content, a finding that directly motivates the scope of this project. Marchal and colleagues (2014) extended lexical analysis to a streaming context, emphasising that features must be cheap enough to compute in real time, a constraint this dissertation adopts as a non-functional requirement.
A recurring tension in the feature-engineering literature concerns the use of external lookups. Features such as domain age, registrar and certificate properties are strongly discriminative but require network queries that introduce latency and external dependencies. Several authors include them freely; others, concerned with real-time deployment, restrict themselves to features computable from the string alone. The present study takes a middle position, permitting only metadata that can be cached locally and refreshed asynchronously, so that the hot path of classification touches no external service.
A broad range of algorithms has been applied to the task. Logistic regression and naive Bayes serve as interpretable baselines whose coefficients can be inspected to understand feature influence. Support vector machines were prominent in earlier work for their strong performance in high-dimensional spaces. Decision trees offer transparency at the cost of a tendency to overfit, a weakness that ensemble methods address directly. Random forests, introduced by Breiman (2001), aggregate many decorrelated trees and have become a de facto standard for tabular phishing-feature data because they combine strong accuracy with resistance to overfitting and a natural measure of feature importance. Gradient-boosted trees, and in particular the scalable XGBoost implementation of Chen and Guestrin (2016), frequently report the strongest results on engineered URL features and are therefore a natural candidate for the top-performing model in a comparative study. Jain and Gupta (2018) reported high accuracy from a client-side machine-learning detector using carefully engineered features, reinforcing the view that the feature set, more than the choice among strong algorithms, drives performance.
A more recent strand dispenses with hand-crafted features altogether and learns representations directly from the raw URL string. Character-level convolutional networks treat the URL as a sequence of characters and learn local patterns indicative of maliciousness; Saxe and Berlin (2017) demonstrated this approach for malicious URL and file detection, and Le and colleagues (2018) extended it with URLNet, which combines character-level and word-level representations. The attraction of these methods is that they can discover patterns a human engineer might miss and adapt to novel obfuscations without manual feature redesign. Their costs are a greater appetite for training data, reduced interpretability and higher inference cost, all of which matter for a low-latency deployment. The present study includes a compact neural model as a comparator precisely to test whether learned representations justify their additional cost on this dataset, while retaining engineered features for the models that must run in the hot path.
Sound evaluation is as important as sound modelling, and the surveyed literature is uneven in this respect. Phishing examples are commonly drawn from community feeds such as PhishTank and OpenPhish, while legitimate examples are drawn from popularity rankings of well-known sites or from curated benign corpora. Two methodological hazards recur. The first is class imbalance handled carelessly: when one class dominates, raw accuracy becomes misleading, and metrics such as precision, recall, F1 and the area under the ROC curve (Fawcett, 2006) are needed to characterise performance honestly. The second, and more insidious, is temporal leakage: when training and test examples are drawn from the same short time window and split at random, the reported accuracy overstates the performance the model would achieve on genuinely future URLs, because the model can exploit campaign-specific regularities that will not persist. A rigorous study must therefore report a full suite of metrics and should, where possible, test generalisation across time rather than only across a random split. The widely used UCI phishing-websites dataset (Dua and Graff, 2019; Mohammad, Thabtah and McCluskey, 2014) has been valuable for comparability but encodes features as pre-computed categorical values, which limits the feature engineering a new study can perform. These observations shape the evaluation design in Chapter 3.
Three gaps emerge from the review. First, while many studies report high accuracy, fewer isolate the contribution of URL-only features computable without any network lookup, which is the regime most relevant to low-latency, privacy-preserving deployment. Second, comparative studies that place interpretable baselines, strong ensembles and a neural comparator on an identical, carefully constructed corpus, reporting not only accuracy but calibration and inference latency, are less common than single-model performance papers. Third, the literature under-reports failure modes: the specific circumstances under which URL-only classifiers err, and the adversarial manipulations that defeat them, are rarely analysed in depth. This dissertation addresses all three gaps by constraining its feature set to the no-lookup regime, by conducting a controlled comparison across algorithm families on one corpus with a full metric suite and latency measurement, and by devoting explicit attention to error analysis and adversarial robustness.
The literature establishes that URL-based machine learning is a mature and effective complement to blacklisting, that feature engineering is the principal determinant of performance, that ensemble tree methods are strong and practical baselines while deep learning offers representation learning at higher cost, and that evaluation practice is frequently weaker than modelling practice. These conclusions directly inform the methodology, system design and evaluation strategy developed in the chapters that follow.
The study adopts a positivist research philosophy and a quantitative, experimental approach. The research questions concern measurable properties of classifiers, the discriminative power of features, the comparative accuracy and latency of algorithms and the characteristics of errors, and are best answered by controlled experiment and statistical analysis rather than by interpretive methods. The approach is deductive in that it derives testable expectations from the literature, chiefly that engineered URL features will support high accuracy, that ensemble methods will outperform linear baselines, and that a neural model will not necessarily justify its additional cost, and then tests those expectations empirically. The work follows the design-science tradition in computing in that it produces and evaluates an artefact, the classification framework, while using the evaluation to generate knowledge about the problem.
Before design could proceed, the framework’s requirements were specified. These are presented as functional and non-functional requirements and were used both to guide design and to structure evaluation.
The evaluation uses an illustrative corpus of approximately 95,000 URLs, constructed to be balanced between the two classes in order to avoid the misleading accuracy figures that unaddressed imbalance produces. The phishing class comprises roughly 47,000 URLs characterised as malicious by publicly documented phishing feeds of the kind described in the literature; the legitimate class comprises roughly 48,000 URLs sampled from reputable popularity rankings and curated benign sources. The slight majority of legitimate examples reflects a deliberate choice to err towards realistic base rates while keeping the classes close enough for balanced learning. The following table summarises the illustrative corpus composition.
| Partition | Phishing URLs | Legitimate URLs | Total |
|---|---|---|---|
| Training (70%) | 32,900 | 33,600 | 66,500 |
| Validation (15%) | 7,050 | 7,200 | 14,250 |
| Test (15%) | 7,050 | 7,200 | 14,250 |
| Total | 47,000 | 48,000 | 95,000 |
Partitioning used stratified sampling to preserve the class ratio in every split, and a fixed random seed to satisfy NFR3. Duplicate URLs and near-duplicates differing only in query ordering were removed before splitting, so that no example could appear in more than one partition and inflate the apparent performance. To probe generalisation beyond a random split, a secondary temporal evaluation was also prepared, in which the test set consisted of URLs collected in a later period than the training set; the implications of this design are returned to in the results and discussion.
Thirty features were defined, all computable from the URL string together with locally cached metadata, in satisfaction of FR2 and NFR2. They fall into the three families established in the literature review. The following table lists a representative selection.
| Family | Feature | Rationale |
|---|---|---|
| Lexical | URL length | Phishing URLs are frequently padded to obscure the true destination. |
| Lexical | Count of dots | Excess subdomaining is used to embed brand names deceptively. |
| Lexical | Presence of at-symbol | The at-symbol causes browsers to ignore the preceding text, enabling spoofing. |
| Lexical | Digit-to-letter ratio | Algorithmically generated hosts tend to contain unusual digit density. |
| Host-based | Raw IP address in host | Legitimate services rarely expose a bare IP to users. |
| Host-based | Number of subdomains | Deep subdomain nesting is a common impersonation tactic. |
| Host-based | Suspicious top-level domain | Certain low-cost TLDs are over-represented among phishing hosts. |
| Host-based | Cached domain age | Phishing domains are typically very recently registered. |
| Derived | Host entropy | High character entropy suggests machine-generated domains. |
| Derived | Security token in path | Words such as “secure” or “login” placed in the path imitate trusted pages. |
Each feature was chosen for a defensible reason grounded in the literature or in attacker tradecraft, rather than being included opportunistically, because unmotivated features add noise and inference cost without reliably improving accuracy. Numeric features were standardised, and the single cached-metadata feature, domain age, was permitted only because it can be pre-fetched and cached, preserving NFR2.
Five algorithms spanning three complexity tiers were selected for comparison, in direct response to RQ2. The interpretable tier comprised logistic regression, chosen for the inspectability of its coefficients. The ensemble tier comprised a random forest (Breiman, 2001) and a gradient-boosted tree model (Chen and Guestrin, 2016), expected from the literature to be the strongest performers on engineered tabular features. The representation-learning tier comprised a compact multilayer perceptron operating on the engineered features and, as a stretch comparator, a character-level convolutional network operating on the raw URL string, following the lineage of Saxe and Berlin (2017) and Le et al. (2018). Including a model at each tier allows the study to test whether added complexity is repaid in accuracy, calibration or robustness.
Model selection and hyperparameter tuning used the training and validation partitions only; the test partition was held out and used exactly once per model, to prevent the optimistic bias that arises from tuning against the test set. Stratified five-fold cross-validation on the training data guided hyperparameter choice. Performance was reported using accuracy, precision, recall, F1 and AUC (Fawcett, 2006), because no single scalar adequately characterises a classifier whose two error types carry different costs: a false negative admits a phishing link, whereas a false positive blocks a legitimate one and erodes user trust. Calibration was assessed because a probability score is only useful for ranking and thresholding if it is well calibrated. Inference latency was measured to verify NFR1. Finally, a feature-importance analysis addressed RQ1 and NFR4, and a structured error analysis addressed RQ3.
The framework was implemented in Python, using the scikit-learn library (Pedregosa et al., 2011) for the linear, forest and perceptron models and feature pipelines, a gradient-boosting library for the boosted trees, and a deep-learning library for the character-level network. Experiments were run on commodity hardware to ensure that the latency measurements reflected a realistic deployment target rather than specialised accelerators. All randomness was seeded, and the code, feature definitions and data manifests were version-controlled to satisfy the reproducibility requirement.
Although the study involves no human participants, several ethical considerations were addressed. The URLs used as data may point to live malicious infrastructure; the framework was therefore designed never to fetch target pages, both for the methodological reason of staying within the no-lookup regime and for the ethical reason of avoiding interaction with attacker-controlled servers. No personal data were collected or processed. Phishing-feed data were used in accordance with the terms of the feeds from which such corpora are conventionally drawn. Finally, the dual-use nature of the research was considered: understanding which features defeat a classifier could in principle inform an attacker. The dissertation mitigates this by discussing evasion at the level of general classes of manipulation rather than by providing a recipe, and by framing the work firmly as defensive.
The framework was designed as a pipeline of loosely coupled modules, a structure chosen to satisfy the extensibility requirement (NFR5) and to make each stage independently testable. Data flows from ingestion through feature extraction into either a training path, which produces a persisted model, or an inference path, which consumes a persisted model to classify new URLs. The separation of training and inference paths is deliberate: training is an offline, resource-intensive activity, whereas inference must satisfy the strict latency budget of NFR1. The following describes each module and the reasoning behind its boundaries.
The ingestion module is responsible for reading raw URL records from source files, normalising them to a canonical form, removing duplicates and near-duplicates, attaching class labels and producing the stratified partitions described in Chapter 3. Canonicalisation is a subtle but important step: two URLs that differ only in the ordering of query parameters, in trailing slashes or in percent-encoding of equivalent characters should be treated as identical, or the model may learn spurious distinctions and the deduplication that prevents leakage will fail. The module therefore applies a consistent normalisation before any downstream processing. By isolating ingestion from feature extraction, the design allows the data source to change, for example from static files to a streaming feed, without disturbing the rest of the pipeline.
The feature extraction module transforms a canonical URL into the fixed-length numeric vector consumed by the models. It is the component most directly governed by the non-functional requirements, because it sits in the inference hot path and must therefore be fast (NFR1) and free of synchronous external calls (NFR2). The module was designed around a registry of feature functions, each a small, pure function from a parsed URL to a numeric value, so that features can be added, removed or reordered declaratively without touching the extraction engine. This design decision was made to honour NFR5 and to make the feature set itself an object of experiment: a study of feature contribution requires the ability to enable and disable features cleanly. The single feature that depends on external information, domain age, is served from a local cache that is populated and refreshed by a background process, so that a cache miss degrades gracefully to a neutral default rather than blocking on a network request.
The training module consumes feature vectors and labels and produces a trained, serialised model together with the fitted preprocessing transforms, for example the scaler used to standardise numeric features. Bundling the preprocessing with the model is a deliberate design choice that prevents a common and damaging error, training-serving skew, in which features are transformed differently at training and inference time. The module exposes a uniform interface across the five algorithms so that they can be trained and compared through identical code paths, which is essential to a fair comparative study: differences in results should reflect the algorithms, not incidental differences in how they were driven. Hyperparameter search, cross-validation and metric computation are orchestrated here against the training and validation partitions only.
The evaluation module applies a persisted model to a held-out partition and computes the full metric suite, the confusion matrix, calibration diagnostics and latency measurements. It was separated from the training module so that a model can be re-evaluated on new data, for example the temporal test set, without retraining, and so that evaluation logic is shared identically across all models. The module also produces the feature-importance output required by NFR4, drawing it from the model where the algorithm supports it natively, as the tree ensembles do, and from a model-agnostic permutation procedure otherwise.
The inference path was designed to be as short as possible. A URL is canonicalised, passed through the feature registry, transformed by the bundled preprocessing and scored by the model, with no step permitted to perform a synchronous network call. The persisted model is loaded once at start-up and held in memory, so that per-URL inference incurs no deserialisation cost. These choices were made specifically to meet NFR1 and to ensure that the latency figures reported in Chapter 6 reflect the design rather than incidental implementation overheads.
The dominant design tension throughout was between accuracy and the operational constraints of latency and independence. A richer feature set incorporating page content and live reputation lookups would almost certainly raise accuracy, but it would violate NFR1 and NFR2 and change the character of the system from a fast first filter to a slow deep inspector. The architecture resolves this tension by positioning the framework explicitly as the first stage of a layered defence: it is designed to be fast, independent and conservative, passing uncertain cases to heavier downstream analysers rather than attempting to be the sole arbiter. This framing, defended further in the discussion, is what makes the chosen trade-offs coherent rather than merely restrictive.
The framework was implemented in Python, a choice justified by the maturity of its machine-learning ecosystem and by the consequent reproducibility and comparability with prior work. The scientific stack provided array computation, data handling and the scikit-learn estimators and pipelines (Pedregosa et al., 2011) used for most models; a dedicated gradient-boosting library provided the boosted-tree implementation (Chen and Guestrin, 2016); and a deep-learning library supported the character-level network. Dependencies were pinned to specific versions and recorded in a manifest, and all stochastic operations were seeded, so that the experiments satisfy the reproducibility requirement (NFR3). The implementation was organised to mirror the module boundaries of Chapter 4, so that the architecture was not merely a diagram but a property of the code.
Preprocessing implemented the ingestion design. URLs were parsed into their components, scheme, host, path, query and fragment, and canonicalised by lower-casing the host, normalising percent-encoding, sorting query parameters and stripping redundant trailing characters. Deduplication was performed on the canonical form, which removed a non-trivial number of records that differed only cosmetically and would otherwise have leaked between partitions. Labels were attached from the source of each record, and the stratified partitioning described in Chapter 3 was applied with a fixed seed. A deliberate implementation decision was to fail loudly on malformed input during ingestion, surfacing data-quality problems early, while failing gracefully on malformed input during inference, returning a conservative default, because an exception in the hot path would be operationally unacceptable.
The feature registry was implemented as a collection of small functions, each returning a single numeric value from a parsed URL, registered under a name and composed by the extraction engine into a fixed-order vector. This structure made it straightforward to compute the feature-ablation experiments reported later, since a feature could be excluded by removing it from the active registry without altering any other code. Care was taken that every function was free of side effects and of network calls; the sole externally informed feature, domain age, read from an in-memory cache and returned a neutral sentinel on a miss. Edge cases absorbed considerable effort: internationalised domain names, extremely long URLs, URLs embedding other URLs in their query strings, and hosts that were valid IP addresses all required explicit handling so that feature values remained well-defined. This attention to edge cases is characteristic of security software, where the inputs are adversarial and malformed by design rather than by accident.
Each model was wrapped behind the uniform training interface so that the comparative experiments drove all five through identical orchestration. The logistic-regression baseline used regularisation selected by cross-validation. The random forest’s number of trees and depth, and the gradient-boosted model’s learning rate, tree depth and number of estimators, were tuned by stratified five-fold cross-validation on the training partition, with the validation partition reserved for final model selection. The multilayer perceptron used a modest architecture appropriate to the thirty-dimensional feature space, with early stopping on validation loss to prevent overfitting. The character-level convolutional network consumed the raw, truncated-and-padded URL string through a learned character embedding, convolutional and pooling layers and a dense classifier, and was trained with early stopping. Preprocessing transforms were fitted on the training partition only and bundled with each model, implementing the anti-skew design decision of Chapter 4.
Hyperparameter selection was treated as an experiment in its own right rather than as an afterthought, because an unfair tuning budget across models would undermine the comparison. Each model received a comparable search effort over its principal hyperparameters, and every candidate configuration was assessed by cross-validation on the training partition so that the validation and test partitions remained uncontaminated. The gradient-boosted model, as anticipated from the literature, was the most sensitive to its learning-rate and depth interaction and benefited most from careful tuning, whereas the random forest was comparatively robust to its settings, a practical advantage in deployment where retuning may be infrequent.
Several challenges arose during implementation and shaped the final design. The most significant was preventing data leakage: an early version deduplicated after partitioning rather than before, allowing cosmetically different copies of the same URL to span the train and test sets and producing implausibly high accuracy that correctly prompted suspicion and investigation. Correcting the order of operations reduced the reported figures to realistic levels, a salutary reminder that an unexpectedly excellent result is more often a bug than a breakthrough. A second challenge was the latency of the character-level network relative to the tree models; while its accuracy was competitive, its inference cost was materially higher, which directly informed the discussion of whether representation learning earns its place in the hot path. A third was the handling of the domain-age cache under misses, resolved by the graceful-default strategy so that the no-lookup requirement was never violated even transiently.
This chapter reports the empirical results of the evaluation designed in Chapter 3. All figures are illustrative, generated to demonstrate correct reporting and interpretation rather than to constitute empirical claims about any operational system. Results are presented in the order of the research questions: overall model performance, feature importance (RQ1), the comparison across algorithm tiers including latency (RQ2), and error analysis with an assessment of robustness (RQ3).
The five models were evaluated on the held-out test partition after selection and tuning on the training and validation partitions. The table below reports the full metric suite. Consistent with the expectations drawn from the literature, the gradient-boosted tree ensemble was the strongest model across every metric, closely followed by the random forest, with the logistic-regression baseline clearly weaker and the two neural models competitive on accuracy but, as discussed below, costlier at inference.
| Model | Accuracy | Precision | Recall | F1 | AUC |
|---|---|---|---|---|---|
| Logistic Regression | 0.918 | 0.911 | 0.924 | 0.917 | 0.962 |
| Multilayer Perceptron | 0.951 | 0.948 | 0.953 | 0.950 | 0.985 |
| Character-Level CNN | 0.959 | 0.956 | 0.961 | 0.958 | 0.990 |
| Random Forest | 0.963 | 0.960 | 0.965 | 0.962 | 0.991 |
| Gradient-Boosted Trees | 0.968 | 0.966 | 0.969 | 0.967 | 0.993 |
The confusion matrix for the leading gradient-boosted model on the 14,250-URL test set is summarised below. Of the 7,050 phishing URLs, an illustrative 6,831 were correctly identified and 219 were missed; of the 7,200 legitimate URLs, 6,972 were correctly passed and 228 were incorrectly flagged. The near-symmetry of the two error types reflects the balanced corpus and the default decision threshold, which the discussion notes could be shifted to trade recall for precision according to operational cost.
| Predicted Phishing | Predicted Legitimate | |
|---|---|---|
| Actual Phishing | 6,831 (true positive) | 219 (false negative) |
| Actual Legitimate | 228 (false positive) | 6,972 (true negative) |
Feature importance was obtained natively from the gradient-boosted model and corroborated with a model-agnostic permutation analysis, the two methods agreeing closely on the ranking of the top features. The five most discriminative features, in descending order of illustrative importance, were the age of the registered domain, the length of the URL, the presence of a raw IP address in the host, the number of subdomains and the misplacement of security-suggestive tokens in the path. The prominence of domain age is consistent with the literature’s observation that phishing domains are typically very recently registered, and it vindicates the decision to retain this single cached-metadata feature despite the engineering cost of the cache. The prominence of the structural lexical features confirms that a great deal of signal is available from the string alone, directly supporting the central hypothesis of the dissertation. Importantly, no single feature dominated to the point of rendering the others redundant; the ablation experiments, in which features were removed one family at a time, showed graceful degradation rather than collapse, indicating that the model draws on complementary signals and is therefore less brittle to the loss of any one of them.
The comparison across tiers tells a clear and practically consequential story. The interpretable baseline, logistic regression, was competent but clearly inferior, confirming that the decision boundary in this feature space is non-linear and that the interactions among features matter. The ensemble tree methods were the strongest, with gradient boosting marginally ahead of the random forest, consistent with the weight of prior work. The neural models matched the ensembles on accuracy but did not surpass them, which is the expected outcome on moderate-sized tabular feature data where tree ensembles are known to be hard to beat. The decisive differentiator was therefore not accuracy but inference latency. The following table reports illustrative mean per-URL inference time on commodity hardware.
| Model | Mean Inference Time per URL | Meets NFR1? |
|---|---|---|
| Logistic Regression | ~0.05 ms | Yes |
| Random Forest | ~0.6 ms | Yes |
| Gradient-Boosted Trees | ~0.4 ms | Yes |
| Multilayer Perceptron | ~0.9 ms | Yes |
| Character-Level CNN | ~6.5 ms | Marginal |
When accuracy, latency and interpretability are weighed together, the gradient-boosted ensemble emerges as the most suitable choice for the hot path: it is the most accurate, comfortably meets the latency budget and exposes native feature importance for auditing. The character-level network, despite respectable accuracy, is an order of magnitude slower and far less interpretable, which, on this dataset, means it does not earn its place in the first line of defence. This is a concrete, evidence-based answer to RQ2 and a reminder that the best model for a problem is defined by its full operating envelope, not by accuracy alone.
Calibration was assessed because the framework returns a probability score intended to support thresholding and triage. The tree ensembles produced reasonably well-calibrated probabilities after a light post-hoc calibration step, meaning that a predicted probability of, say, 0.8 corresponded to approximately an 80 per cent empirical likelihood of phishing. Good calibration is operationally valuable because it allows a deployment to set a meaningful threshold and to route borderline cases, those near 0.5, to heavier downstream analysis rather than forcing a premature binary verdict. The logistic model was naturally well calibrated but less accurate; the neural models required more substantial calibration adjustment.
The secondary temporal evaluation, in which the test URLs were drawn from a later period than the training URLs, produced a modest but clear decline in performance relative to the random split, with illustrative accuracy for the leading model falling from 0.968 to approximately 0.94. This decline is the empirical signature of concept drift, and its presence despite a strong random-split result is one of the most important findings of the study. It demonstrates that the headline figure obtained from a random split, while not wrong, is optimistic as an estimate of sustained operational accuracy, and it motivates the retraining regime discussed in the next chapter. A study that reported only the random-split figure would have painted a materially more flattering picture than the data support.
Qualitative analysis of the misclassified URLs revealed systematic patterns rather than random noise. False negatives, phishing URLs judged legitimate, clustered into two groups: links hosted on compromised but otherwise reputable domains, where the host-based features legitimately pointed towards benignity, and links shortened by reputable URL-shortening services, where the true destination was hidden from the string entirely. Both groups expose a fundamental limit of URL-only analysis: when the string genuinely lacks the signal, no amount of modelling can recover it. False positives, legitimate URLs judged phishing, clustered among long, parameter-heavy URLs produced by legitimate web applications, and among newly registered legitimate domains, which share the recency signal that phishing domains exhibit. These patterns are not merely incidental; they define the boundary of the technique and feed directly into the discussion of where the framework must defer to other layers of defence.
The results support the central hypothesis that URL-only machine learning can classify phishing links with high accuracy and low latency: the leading model reached an illustrative 96.8 per cent accuracy at well under a millisecond per URL, using features computable without any synchronous network lookup. They also qualify that hypothesis in two important ways: performance degrades under temporal drift, and a structured class of URLs, those lacking signal in the string, lies beyond the technique’s reach. These findings, both the positive and the qualifying, are taken up in the discussion.
The three research questions can now be answered directly from the evidence. In answer to RQ1, the most discriminative URL-derived features were domain age, URL length, the presence of a raw IP in the host, the number of subdomains and the misplacement of security tokens; crucially, the signal was distributed across complementary features rather than concentrated in one, which has favourable implications for robustness. In answer to RQ2, models of differing complexity converged on similar accuracy at the top of the range, with tree ensembles leading, but diverged sharply on latency and interpretability, so that the gradient-boosted ensemble is the rational choice for the hot path while the character-level network, despite its sophistication, is not. In answer to RQ3, the principal failure modes are structural: compromised reputable hosts and shortened URLs defeat the classifier because the string lacks the necessary signal, while long legitimate URLs and newly registered legitimate domains generate false positives. These answers are coherent with one another and with the design philosophy that positioned the framework as a fast first filter rather than a sole arbiter.
The findings corroborate and extend the surveyed literature in several respects. They confirm the field’s recurring conclusion that URL-only features support high accuracy (Sahingoz et al., 2019) and that tree ensembles are strong, practical performers on engineered features (Jain and Gupta, 2018). They also confirm a subtler point that is sometimes under-emphasised: that on moderate-sized tabular feature data, learned representations from a character-level network (Saxe and Berlin, 2017; Le et al., 2018) do not necessarily outperform well-engineered features fed to an ensemble, and may carry a latency penalty that disqualifies them from real-time use. Perhaps most importantly, by implementing the temporal evaluation that much of the literature omits, the study provides concrete evidence for the concept-drift concern that surveys such as Sahoo, Liu and Hoi (2017) raise in principle, translating a known caveat into a measured effect. In this respect the dissertation addresses the evaluation-practice gap identified in the literature review, not merely the modelling gap.
The practical message of the study is that a URL-only classifier should be deployed as the first stage of a layered architecture, not as a standalone solution. Its speed and independence make it ideal for triaging the overwhelming majority of links cheaply, confidently passing clear cases and confidently blocking obvious ones, while routing the uncertain middle, and the structurally hard cases such as shortened URLs, to slower and richer analysers that may fetch content or consult reputation services. The good calibration of the ensemble models is what makes this triage design feasible, because a well-calibrated probability is precisely what is needed to define the uncertain band. The adjustable decision threshold further allows a deployment to tune the balance between missing phishing links and inconveniencing users according to its own cost model, for example biasing towards recall in a high-risk enterprise setting and towards precision in a consumer product where false positives are especially corrosive to trust.
Any honest discussion of a security classifier must confront the adversary. Because the features are known in kind, an attacker can in principle craft URLs that minimise the signals the model relies upon: shortening the URL, avoiding raw IPs, ageing domains before use and avoiding conspicuous token misplacement. The error analysis already shows that shortened URLs are a live weakness. This adversarial pressure is not a reason to abandon the approach but a reason to frame it correctly: the classifier raises the cost and reduces the convenience of phishing at scale, and its value lies in catching the large volume of unsophisticated, automatically generated URLs rather than in defeating a determined, well-resourced adversary single-handedly. Robustness is best pursued through the layered architecture, through periodic retraining on fresh data to blunt drift and evasion, and through combining the classifier with signals the attacker cannot easily control, such as the behaviour of the wider campaign, which lie outside the scope of this study but define sensible future work. The dissertation deliberately discusses evasion at the level of general classes rather than providing an operational recipe, in keeping with the ethical considerations set out earlier.
The study’s limitations must be stated plainly. First, the corpus is a static snapshot; although the temporal evaluation probes drift, it cannot substitute for sustained measurement against a live feed, and the reported figures are illustrative rather than operational. Second, the no-lookup constraint, while central to the research question, caps achievable accuracy by excluding genuinely informative content and reputation signals; the framework is therefore not the most accurate possible detector but the most accurate within a deliberately restricted regime. Third, the labels inherited from phishing feeds and popularity rankings are imperfect, and label noise places a ceiling on measurable accuracy. Fourth, the single cached-metadata feature, domain age, introduces an operational dependency on a cache that, while off the hot path, must be maintained. Recognising these limitations is not a weakness of the study but a condition of interpreting its results responsibly.
This dissertation set out to determine how far machine learning can classify phishing URLs using only features derivable from the URL string and lightweight local metadata, and to establish the conditions under which such an approach succeeds and fails. It reviewed the literature to locate a defensible gap, specified requirements for a low-latency, independent detector, designed and implemented a modular framework, and evaluated five algorithms across three complexity tiers on an illustrative balanced corpus using a full suite of metrics, a temporal generalisation test, feature-importance analysis and structured error analysis. The leading gradient-boosted ensemble achieved an illustrative 96.8 per cent accuracy, an F1 of 0.967 and an AUC of 0.993 at sub-millisecond latency, confirming the central hypothesis while the temporal and error analyses delineated its limits.
The dissertation makes three contributions. Methodologically, it demonstrates a controlled comparison of algorithm tiers on a single corpus that reports not only accuracy but calibration, latency and temporal generalisation, addressing an evaluation-practice gap in the field. Substantively, it isolates the performance attainable in the strict no-lookup regime and identifies the feature signals that drive it, answering the question of how much can be achieved from the string alone. Practically, it articulates and evidences the case for positioning a URL-only classifier as the fast first stage of a layered defence, with calibrated probabilities enabling principled triage. Taken together these contributions offer both a reusable method and a clear, honest account of what URL-only classification can and cannot do.
Several avenues extend the work naturally. The most pressing is live, longitudinal evaluation against a streaming feed, which would replace the illustrative snapshot figures with sustained operational measurements and quantify the retraining cadence needed to counter drift. A second avenue is the integration of the classifier into an explicit layered pipeline, measuring the end-to-end accuracy and latency of a system that triages with the URL model and escalates uncertain cases to content and reputation analysers. A third is a systematic study of adversarial robustness, measuring how performance degrades under defined classes of evasion and evaluating defensive techniques such as adversarial training and feature hardening, conducted within a responsible-disclosure ethical framework. A fourth is to expand the deep-learning comparison with modern sequence models, testing whether richer representations close the gap on structurally hard cases such as shortened URLs, perhaps by learning to flag rather than resolve them. Finally, applying the framework beyond phishing to the broader problem of malicious-URL detection would test the generality of the feature set and the architecture.
Phishing endures because it exploits human trust at machine scale, and no single technique will eliminate it. The contribution of a fast, independent, well-understood URL classifier is not to be a silver bullet but to raise the baseline cheaply and reliably, intercepting the great volume of commodity attacks and freeing heavier defences to concentrate on the hard residue. This dissertation has shown that such a classifier is achievable, has characterised honestly where it succeeds and where it must defer, and has set out a credible path for strengthening it. In a defensive landscape where speed, independence and interpretability are often as valuable as raw accuracy, those qualities make URL-only machine learning a durable component of layered protection rather than a passing expedient.
Abdelhamid, N., Ayesh, A. and Thabtah, F. (2014) ‘Phishing detection based associative classification data mining’, Expert Systems with Applications, 41(13), pp. 5948–5959.
APWG (2023) Phishing Activity Trends Report. Anti-Phishing Working Group. Available at: https://apwg.org/trendsreports/ (Accessed: 20 October 2026).
Breiman, L. (2001) ‘Random forests’, Machine Learning, 45(1), pp. 5–32.
Chen, T. and Guestrin, C. (2016) ‘XGBoost: a scalable tree boosting system’, in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, pp. 785–794.
Dua, D. and Graff, C. (2019) UCI Machine Learning Repository. Irvine, CA: University of California, School of Information and Computer Science. Available at: https://archive.ics.uci.edu/ml (Accessed: 20 October 2026).
Fawcett, T. (2006) ‘An introduction to ROC analysis’, Pattern Recognition Letters, 27(8), pp. 861–874.
Jain, A.K. and Gupta, B.B. (2018) ‘Towards detection of phishing websites on client-side using machine learning based approach’, Telecommunication Systems, 68(4), pp. 687–700.
Le, H., Pham, Q., Sahoo, D. and Hoi, S.C.H. (2018) URLNet: learning a URL representation with deep learning for malicious URL detection. arXiv:1802.03162.
Marchal, S., Francois, J., State, R. and Engel, T. (2014) ‘PhishStorm: detecting phishing with streaming analytics’, IEEE Transactions on Network and Service Management, 11(4), pp. 458–471.
Mohammad, R.M., Thabtah, F. and McCluskey, L. (2014) ‘Predicting phishing websites based on self-structuring neural network’, Neural Computing and Applications, 25(2), pp. 443–458.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M. and Duchesnay, E. (2011) ‘Scikit-learn: machine learning in Python’, Journal of Machine Learning Research, 12, pp. 2825–2830.
Sahingoz, O.K., Buber, E., Demir, O. and Diri, B. (2019) ‘Machine learning based phishing detection from URLs’, Expert Systems with Applications, 117, pp. 345–357.
Sahoo, D., Liu, C. and Hoi, S.C.H. (2017) Malicious URL detection using machine learning: a survey. arXiv:1701.07179.
Saxe, J. and Berlin, K. (2017) eXpose: a character-level convolutional neural network with embeddings for detecting malicious URLs, file paths and registry keys. arXiv:1702.08568.
You May Also Like
All work is written by human writers. 100% AI free, guaranteed.
100% money back guarantee if you find plagiarism in our work.
COMPANY DETAILS
