前一篇使用 C# 測試了 Cloudflare Clef 的 XSS 判斷,這次想直接把 Clef Flash 放進來比較,最近幫助一些朋友,最近駭客事件真的很多..

之後我發現有 Clef-Flash 我就在想是不是能夠更加的大規模使用畢竟他便宜很多
而且感覺也比較快速,這兩個光是模型量就不同 Clef (27B) , Clef-Flash(9B)
我自己比較在意的其實不只是模型參數量,而是同樣一筆資料丟進 API 之後,判斷機率、回應速度跟花費究竟差多少,所以就寫了一個 .NET 10 Benchmark,拿 100 筆 XSS 測試字串連續跑 5 輪,並且統計如果其中之一小於 90% 的參數都列出來
1. 這次測試怎麼跑
兩個模型分別是 @cf/cloudflare/clef 與 @cf/cloudflare/clef-flash
每一輪用同樣 100 筆 XSS Payload,兩個模型各呼叫 100 次,總共 5 輪,也就是 1,000 次正式 API Request。正式測試前各跑 5 次 Warm-up,不納入統計
Payload 順序每輪打亂,對同一筆資料也會隨機決定先呼叫哪個模型
每次 Request 後仍然保留 Task.Delay(200),主要是避免測試太密集,不過這段 Sleep 不會計算在模型的 API 總耗時裡面
Instructions 也是兩個模型共用同一份,要求判斷是否含有可執行 JavaScript、HTML Event Handler、javascript: URL 等 XSS 執行特徵,普通 HTML 不應該直接判成有風險。
2. 五輪實際測試結果
| Run | Clef API 總耗時 | Clef Flash API 總耗時 | 較快模型 |
|---|---|---|---|
| 五輪合計 | 220.18 秒 | 147.59 秒 | Flash 快約 1.49 倍 |
| 1 | 40.18 秒 | 36.36 秒 | Clef Flash |
| 2 | 51.33 秒 | 27.64 秒 | Clef Flash |
| 3 | 42.02 秒 | 27.69 秒 | Clef Flash |
| 4 | 45.65 秒 | 22.46 秒 | Clef Flash |
| 5 | 40.99 秒 | 33.44 秒 | Clef Flash |
這次結果反而跟我一開始測的方向不同,調整測試方式、加入 Warm-up,也把 Sleep 排除之後,Clef Flash 五輪全部比 Clef 快;五輪合計少了 72.58 秒
不過 Flash 也不是每個地方都比較快,像 P95 反而比 Clef 慢,這個我覺得還滿有意思的。
| 項目 | Clef | Clef Flash |
|---|---|---|
| 成功 / 正式呼叫 | 500 / 500 | 500 / 500 |
| 平均 API 延遲 | 440.35 ms | 295.18 ms |
| P50 | 344.50 ms | 148.00 ms |
| P95 | 889.15 ms | 1082.80 ms |
| P99 | 1203.64 ms | 1196.63 ms |
| Input Tokens | 103,020 | 103,020 |
| 平均 XSS 分數 | 97.91% | 91.07% |
| 低於 90% 次數 | 5 / 500 | 80 / 500 |
| Retry | 0 | 0 |
| 估算費用 | US$0.024725 | US$0.003915 |
這次每個模型 500 筆 Request 都用了 103,020 個 Input Tokens,平均每次 206.04 個。Cloudflare 官方目前 Clef 是每百萬 Input Tokens US$0.240,Clef Flash 是 US$0.038,輸出 Token 不另外計費,所以同樣 Token 數量,Flash 的理論費用就少了 84.17%。
依照這次的每次 Request 規模,推算 100 萬次呼叫,Clef 約 US$49.45,Flash 約 US$7.83。這是依官方輸入 Token 單價估算,還不是 Cloudflare 帳單實際扣款,其他免費額度和服務費先不算進去。
3. 判斷分數也有明顯差異
速度跟價格雖然是 Flash 比較有優勢,但兩個模型判斷同一筆 XSS Payload 的機率差得還滿多。以 90% 當低分門檻,Clef 每輪只有 1 筆低於 90%,Flash 每輪都有 16 筆。因為同一批 Payload 在五輪的回傳分數完全一樣,所以最後就是 5 次對 80 次。
| Payload | Clef | Flash |
|---|---|---|
${alert(1)} | 98.06% | 17.72% |
<svg><animate onbegin="alert(1)" attributeName="x" dur="1s"/></svg> | 97.91% | 30.34% |
<details open ontoggle=alert(1)>x</details> | 98.62% | 60.42% |
<script>alert(1)</script> | 69.25% | 95.57% |
像 ${alert(1)} 這就比較有問題 不過這種字串本身還是要放在對的 JavaScript Template Literal Context 才能討論會不會執行
下面附的是這次實際測試用的三份檔案:C# Benchmark 主程式, 100 筆 XSS Payload,最後是完整執行結果 output.txt
C# 主程式:
internal class ClefXssBenchmarkTest
{
private const double ClefPricePerMillion = 0.240;
private const double FlashPricePerMillion = 0.038;
private static readonly string[] Models = { "clef", "clef-flash" };
private const string Instructions =
"Does this input contain executable JavaScript, " +
"HTML event handlers, javascript: URLs, or other " +
"clear XSS execution patterns? " +
"Ordinary HTML tags without script execution " +
"must be classified as No.";
private const double LowThreshold = 0.90;
private const int Runs = 5;
private const int WarmupCount = 5;
private const int RetryMax = 2;
private const int SleepMs = 200;
public static async Task RunAsync()
{
Console.OutputEncoding = Encoding.UTF8;
var accountId = Environment.GetEnvironmentVariable("CLOUDFLARE_ACCOUNT_ID");
var apiToken = Environment.GetEnvironmentVariable("CLOUDFLARE_API_TOKEN");
if (string.IsNullOrWhiteSpace(accountId) || string.IsNullOrWhiteSpace(apiToken))
{
Console.WriteLine("缺少環境變數:請設定 CLOUDFLARE_ACCOUNT_ID 與 CLOUDFLARE_API_TOKEN 後再執行。");
return;
}
var payloads = LoadPayloads();
if (payloads.Count == 0)
{
Console.WriteLine("找不到 xss-benchmark-100.txt 或檔案為空。");
return;
}
// 同步輸出到 output.txt
var outputPath = Path.Combine(Directory.GetCurrentDirectory(), "output.txt");
using var fileWriter = new StreamWriter(outputPath, false, new UTF8Encoding(false));
var tee = new TeeTextWriter(Console.Out, fileWriter);
Console.SetOut(tee);
Console.WriteLine($"載入 {payloads.Count} 筆 XSS 測試樣本");
Console.WriteLine($"測試模型:{string.Join(", ", Models)}");
Console.WriteLine($"低分門檻:< {LowThreshold:P0}");
Console.WriteLine($"執行次數:{Runs}");
Console.WriteLine($"Warm-up 次數:每個模型 {WarmupCount} 次(不納入統計)");
Console.WriteLine();
using var client = new HttpClient { Timeout = TimeSpan.FromSeconds(120) };
client.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("Bearer", apiToken);
// Warm-up
Console.WriteLine("開始 Warm-up...");
foreach (var model in Models)
{
var endpoint = $"https://api.cloudflare.com/client/v4/accounts/{accountId}/ai/run/@cf/cloudflare/{model}";
for (int w = 0; w < WarmupCount; w++)
{
await CallAsync(client, endpoint, model, payloads[0], new List<string>());
}
Console.WriteLine($" {model} Warm-up 完成 ({WarmupCount} 次)");
}
Console.WriteLine("Warm-up 結束,正式測試開始。");
Console.WriteLine();
var runRecords = new List<RunRecord>();
var overallTimer = Stopwatch.StartNew();
var rng = new Random();
for (int run = 1; run <= Runs; run++)
{
Console.WriteLine(new string('#', 70));
Console.WriteLine($"第 {run} / {Runs} 輪 Benchmark");
Console.WriteLine(new string('#', 70));
Console.WriteLine();
var runTimer = Stopwatch.StartNew();
var results = new Dictionary<string, List<ResultRow>>();
foreach (var m in Models) results[m] = new List<ResultRow>();
var retryWait = new Dictionary<string, long> { [Models[0]] = 0, [Models[1]] = 0 };
var retryCount = new Dictionary<string, int> { [Models[0]] = 0, [Models[1]] = 0 };
var retryDetails = new Dictionary<string, List<string>> { [Models[0]] = new List<string>(), [Models[1]] = new List<string>() };
// 隨機化 Payload 順序
var order = Enumerable.Range(0, payloads.Count).OrderBy(_ => rng.Next()).ToList();
foreach (var idx in order)
{
var payload = payloads[idx];
// 隨機決定模型執行先後
bool clefFirst = rng.Next(2) == 0;
var modelOrder = clefFirst
? new[] { Models[0], Models[1] }
: new[] { Models[1], Models[0] };
foreach (var model in modelOrder)
{
var endpoint = $"https://api.cloudflare.com/client/v4/accounts/{accountId}/ai/run/@cf/cloudflare/{model}";
var res = await CallAsync(client, endpoint, model, payload, retryDetails[model]);
results[model].Add(new ResultRow(idx + 1, payload, res.Probability, res.Ms, res.Tokens, res.Status, res.RetryCount, res.RetryWaitMs));
retryCount[model] += res.RetryCount;
retryWait[model] += res.RetryWaitMs;
await Task.Delay(SleepMs);
}
}
runTimer.Stop();
// 收集低分樣本
var a = results[Models[0]];
var b = results[Models[1]];
var lowSamples = new List<LowSample>();
for (int i = 0; i < Math.Min(a.Count, b.Count); i++)
{
bool la = a[i].Probability.HasValue && a[i].Probability.Value < LowThreshold;
bool lb = b[i].Probability.HasValue && b[i].Probability.Value < LowThreshold;
if (la || lb)
{
lowSamples.Add(new LowSample(a[i].Index, a[i].Payload, a[i].Probability, b[i].Probability));
}
}
// 依原始編號排序結果
foreach (var m in Models)
results[m] = results[m].OrderBy(r => r.Index).ToList();
var clefApiMs = results[Models[0]].Sum(r => r.Ms);
var flashApiMs = results[Models[1]].Sum(r => r.Ms);
runRecords.Add(new RunRecord(
run, runTimer.ElapsedMilliseconds, clefApiMs, flashApiMs,
results, retryCount, retryWait, lowSamples));
// 印出本輪報表
PrintRunReport(run, results, retryCount, retryWait, lowSamples, SleepMs);
}
overallTimer.Stop();
// 最終總覽
PrintFinalSummary(runRecords, overallTimer, accountId, apiToken);
// Retry 詳細記錄
Console.WriteLine();
Console.WriteLine(new string('=', 70));
Console.WriteLine("========== Retry 詳細記錄 ==========");
Console.WriteLine(new string('=', 70));
bool anyRetry = false;
foreach (var rec in runRecords)
{
foreach (var model in Models)
{
if (rec.RetryCount[model] > 0)
{
anyRetry = true;
Console.WriteLine($"Run {rec.Run} {model}: retries={rec.RetryCount[model]}, wait={rec.RetryWaitMs[model]} ms");
}
}
}
if (!anyRetry) Console.WriteLine("(無 Retry)");
Console.WriteLine();
Console.WriteLine($"output.txt 已輸出:{outputPath}");
tee.Flush();
}
private static void PrintRunReport(
int run,
Dictionary<string, List<ResultRow>> results,
Dictionary<string, int> retryCount,
Dictionary<string, long> retryWait,
List<LowSample> lowSamples,
int sleepMs)
{
Console.WriteLine($"模型 {Models[0],14} {Models[1],14}");
Console.WriteLine(new string('-', 58));
foreach (var model in Models)
{
// placeholder for alignment handled below
}
// Build per-model metrics
var metrics = new Dictionary<string, ModelMetrics>();
foreach (var model in Models)
{
var rows = results[model];
var success = rows.Where(r => r.Status == "OK").ToList();
var latencies = success.Select(r => r.Ms).OrderBy(x => x).ToList();
var tokens = rows.Where(r => r.Tokens.HasValue).Select(r => r.Tokens!.Value).ToList();
int? totalTokens = tokens.Count > 0 ? tokens.Sum() : (int?)null;
metrics[model] = new ModelMetrics
{
RequestCount = rows.Count,
SuccessCount = success.Count,
FailCount = rows.Count - success.Count,
TotalTokens = totalTokens,
ApiTotalMs = rows.Sum(r => r.Ms),
AvgLatency = latencies.Count > 0 ? latencies.Average() : (double?)null,
MinLatency = latencies.Count > 0 ? latencies.Min() : (long?)null,
MaxLatency = latencies.Count > 0 ? latencies.Max() : (long?)null,
P50 = Percentile(latencies, 50),
P95 = Percentile(latencies, 95),
P99 = Percentile(latencies, 99),
RetryCount = retryCount[model],
RetryWaitMs = retryWait[model],
SleepMs = rows.Count * sleepMs,
AvgProb = success.Count > 0 ? success.Average(r => r.Probability!.Value) : (double?)null,
LowCount = rows.Count(r => r.Probability.HasValue && r.Probability.Value < LowThreshold),
};
}
// Print table rows
void Row(string label, Func<string, string> val)
{
Console.WriteLine($"{label,-26} {val(Models[0]),14} {val(Models[1]),14}");
}
Row("Request 數量", m => metrics[m].RequestCount.ToString());
Row("成功數量", m => metrics[m].SuccessCount.ToString());
Row("失敗數量", m => metrics[m].FailCount.ToString());
Row("Input Tokens", m => metrics[m].TotalTokens.HasValue ? metrics[m].TotalTokens!.Value.ToString("N0") : "N/A");
Row("API 總耗時 (ms)", m => metrics[m].ApiTotalMs.ToString("N0"));
Row("API 總耗時 (秒)", m => (metrics[m].ApiTotalMs / 1000.0).ToString("F2"));
Row("平均 API 延遲 (ms)", m => metrics[m].AvgLatency.HasValue ? metrics[m].AvgLatency!.Value.ToString("F2") : "N/A");
Row("P50 (ms)", m => metrics[m].P50.HasValue ? metrics[m].P50!.Value.ToString("F2") : "N/A");
Row("P95 (ms)", m => metrics[m].P95.HasValue ? metrics[m].P95!.Value.ToString("F2") : "N/A");
Row("P99 (ms)", m => metrics[m].P99.HasValue ? metrics[m].P99!.Value.ToString("F2") : "N/A");
Row("最短 API 延遲 (ms)", m => metrics[m].MinLatency.HasValue ? metrics[m].MinLatency!.Value.ToString() : "N/A");
Row("最長 API 延遲 (ms)", m => metrics[m].MaxLatency.HasValue ? metrics[m].MaxLatency!.Value.ToString() : "N/A");
Row("Retry 次數", m => metrics[m].RetryCount.ToString());
Row("Retry 等待時間 (ms)", m => metrics[m].RetryWaitMs.ToString("N0"));
Row("固定 Sleep 時間 (ms)", m => metrics[m].SleepMs.ToString("N0"));
Row("XSS 平均機率 (%)", m => metrics[m].AvgProb.HasValue ? (metrics[m].AvgProb!.Value * 100).ToString("F2") : "N/A");
Row("低於 90% 數量", m => metrics[m].LowCount.ToString());
Row("預估費用 (USD)", m => FormatCost(metrics[m].TotalTokens, m));
Console.WriteLine();
Console.WriteLine("本輪 API 耗時比較:");
var clefMs = metrics[Models[0]].ApiTotalMs;
var flashMs = metrics[Models[1]].ApiTotalMs;
Console.WriteLine($"Clef API 總耗時:{clefMs} ms ({clefMs / 1000.0:F2} s)");
Console.WriteLine($"Clef Flash API 總耗時:{flashMs} ms ({flashMs / 1000.0:F2} s)");
string faster = clefMs <= flashMs ? "Clef" : "Clef Flash";
long diff = Math.Abs(clefMs - flashMs);
double ratio = (double)Math.Max(clefMs, flashMs) / Math.Max(1, Math.Min(clefMs, flashMs));
Console.WriteLine($"較快模型:{faster}");
Console.WriteLine($"總耗時差距:{diff} ms ({diff / 1000.0:F2} s)");
Console.WriteLine($"速度倍率:{ratio:F2} x");
Console.WriteLine();
Console.WriteLine("注意:API 總耗時不包含任何 Sleep、Task.Delay、");
Console.WriteLine("Retry Backoff 或 Warm-up。");
Console.WriteLine();
// 低分 Payload 明細
if (lowSamples.Count > 0)
{
Console.WriteLine($"低分 Payload 明細(< {LowThreshold:P0},共 {lowSamples.Count} 筆):");
Console.WriteLine($"{"#",4} {"Clef",8} {"Flash",8} {"Diff",8} Payload");
Console.WriteLine(new string('-', 90));
foreach (var ls in lowSamples.OrderBy(x => x.Index))
{
var pa = ls.ClefProb.HasValue ? ls.ClefProb.Value.ToString("P2") : "N/A";
var pb = ls.FlashProb.HasValue ? ls.FlashProb.Value.ToString("P2") : "N/A";
string diffStr = (ls.ClefProb.HasValue && ls.FlashProb.HasValue)
? ((ls.ClefProb.Value - ls.FlashProb.Value) * 100).ToString("F2") + " pp"
: "N/A";
Console.WriteLine($"{ls.Index,4} {pa,8} {pb,8} {diffStr,8} {ls.Payload}");
}
Console.WriteLine();
}
// 百分位數計算方式說明
Console.WriteLine("百分位數計算方式:成功 Request 的耗時排序後,採線性插值法 (P50/P95/P99)。");
Console.WriteLine("API 總耗時 = 所有 HTTP 嘗試(含失敗)的端到端耗時加總,不含 Sleep/Backoff/Warm-up。");
Console.WriteLine();
}
private static void PrintFinalSummary(List<RunRecord> runRecords, Stopwatch overallTimer, string accountId, string apiToken)
{
Console.WriteLine();
Console.WriteLine(new string('=', 70));
Console.WriteLine("CLEF VS CLEF FLASH - FINAL BENCHMARK");
Console.WriteLine(new string('=', 70));
Console.WriteLine();
Console.WriteLine($"{"輪次",8} {"Clef API 總秒數",16} {"Flash API 總秒數",18} {"較快模型",12}");
Console.WriteLine(new string('-', 58));
long totalClef = 0, totalFlash = 0;
int clefWins = 0, flashWins = 0;
foreach (var rec in runRecords)
{
var cMs = rec.Results[Models[0]].Sum(r => r.Ms);
var fMs = rec.Results[Models[1]].Sum(r => r.Ms);
totalClef += cMs;
totalFlash += fMs;
string winner = cMs <= fMs ? "Clef" : "Clef Flash";
if (cMs <= fMs) clefWins++; else flashWins++;
Console.WriteLine($"Run {rec.Run,3} {cMs / 1000.0,16:F2} {fMs / 1000.0,18:F2} {winner,12}");
}
Console.WriteLine(new string('-', 58));
Console.WriteLine($"{"五輪合計",8} {totalClef / 1000.0,16:F2} {totalFlash / 1000.0,18:F2} {"",12}");
Console.WriteLine();
long totalDiff = Math.Abs(totalClef - totalFlash);
double totalRatio = (double)Math.Max(totalClef, totalFlash) / Math.Max(1, Math.Min(totalClef, totalFlash));
string totalFaster = totalClef <= totalFlash ? "Clef" : "Clef Flash";
Console.WriteLine($"五輪總 API 耗時差距:{totalDiff} ms ({totalDiff / 1000.0:F2} 秒)");
Console.WriteLine($"五輪總 API 速度倍率:{totalRatio:F2} x({totalFaster} 較快)");
Console.WriteLine($"Clef 勝出輪數:{clefWins},Clef Flash 勝出輪數:{flashWins}");
Console.WriteLine();
// Tokens
long clefTokens = runRecords.SelectMany(r => r.Results[Models[0]]).Where(r => r.Tokens.HasValue).Sum(r => r.Tokens!.Value);
long flashTokens = runRecords.SelectMany(r => r.Results[Models[1]]).Where(r => r.Tokens.HasValue).Sum(r => r.Tokens!.Value);
bool clefHasTokens = runRecords.SelectMany(r => r.Results[Models[0]]).Any(r => r.Tokens.HasValue);
bool flashHasTokens = runRecords.SelectMany(r => r.Results[Models[1]]).Any(r => r.Tokens.HasValue);
Console.WriteLine($"Clef 總 Input Tokens:{(clefHasTokens ? clefTokens.ToString("N0") : "N/A")}");
Console.WriteLine($"Clef Flash 總 Input Tokens:{(flashHasTokens ? flashTokens.ToString("N0") : "N/A")}");
Console.WriteLine();
// Costs
double? clefCost = clefHasTokens ? clefTokens * ClefPricePerMillion / 1_000_000.0 : (double?)null;
double? flashCost = flashHasTokens ? flashTokens * FlashPricePerMillion / 1_000_000.0 : (double?)null;
Console.WriteLine($"Clef 總預估成本:{(clefCost.HasValue ? "US$" + clefCost.Value.ToString("F6") : "N/A")}");
Console.WriteLine($"Clef Flash 總預估成本:{(flashCost.HasValue ? "US$" + flashCost.Value.ToString("F6") : "N/A")}");
if (clefCost.HasValue && flashCost.HasValue && clefCost.Value > 0)
{
double savings = (1 - flashCost.Value / clefCost.Value) * 100;
Console.WriteLine($"成本節省比例:{savings:F2}%");
}
Console.WriteLine();
// Per-1M-requests estimate based on avg tokens per request
long clefReqCount = runRecords.Count * 100;
long flashReqCount = runRecords.Count * 100;
double clefAvgTokens = clefHasTokens ? (double)clefTokens / clefReqCount : 0;
double flashAvgTokens = flashHasTokens ? (double)flashTokens / flashReqCount : 0;
Console.WriteLine($"平均每 Request Input Tokens:Clef {clefAvgTokens:F2},Clef Flash {flashAvgTokens:F2}");
Console.WriteLine($"每 100 萬次相同規模請求預估成本:Clef US${(clefAvgTokens * ClefPricePerMillion):F2},Clef Flash US${(flashAvgTokens * FlashPricePerMillion):F2}");
Console.WriteLine();
// Aggregate stats
Console.WriteLine(new string('=', 70));
Console.WriteLine("五輪彙總");
Console.WriteLine(new string('=', 70));
foreach (var model in Models)
{
var allRows = runRecords.SelectMany(r => r.Results[model]).ToList();
var success = allRows.Where(r => r.Status == "OK").ToList();
var latencies = success.Select(r => r.Ms).OrderBy(x => x).ToList();
int low = allRows.Count(r => r.Probability.HasValue && r.Probability.Value < LowThreshold);
double avgProb = success.Count > 0 ? success.Average(r => r.Probability!.Value) : 0;
int retries = runRecords.Sum(r => r.RetryCount[model]);
long retryWait = runRecords.Sum(r => r.RetryWaitMs[model]);
Console.WriteLine($"{model}: OK={success.Count}/{allRows.Count}, 低分={low}, 平均機率={avgProb:P2}, " +
$"平均延遲={ (latencies.Count>0?latencies.Average():0):F2} ms, " +
$"P50={Percentile(latencies,50):F2}, P95={Percentile(latencies,95):F2}, P99={Percentile(latencies,99):F2}, " +
$"重試={retries}, 重試等待={retryWait} ms");
}
Console.WriteLine();
overallTimer.Stop();
Console.WriteLine($"整體牆鐘時間:{overallTimer.ElapsedMilliseconds} ms ({overallTimer.Elapsed.TotalSeconds:F2} s)");
}
private static string FormatCost(int? totalTokens, string model)
{
if (!totalTokens.HasValue) return "N/A";
double price = model == "clef" ? ClefPricePerMillion : FlashPricePerMillion;
double cost = totalTokens.Value * price / 1_000_000.0;
return "US$" + cost.ToString("F6");
}
private static double? Percentile(List<long> sorted, int p)
{
if (sorted.Count == 0) return null;
if (sorted.Count == 1) return sorted[0];
double pos = (p / 100.0) * (sorted.Count - 1);
int lo = (int)Math.Floor(pos);
int hi = (int)Math.Ceiling(pos);
if (lo == hi) return sorted[lo];
double frac = pos - lo;
return sorted[lo] + (sorted[hi] - sorted[lo]) * frac;
}
private static async Task<CallResult> CallAsync(
HttpClient client, string endpoint, string model, string payload, List<string> retryLog)
{
var request = new
{
model = model,
state = new
{
source = "Website user input",
input = payload,
context = "Untrusted text that may be rendered as HTML"
},
questions = new
{
xss = new
{
type = "noul",
instructions = Instructions
}
}
};
var attemptTimes = new List<long>();
int retryCount = 0;
long retryWaitMs = 0;
var details = new List<string>();
for (int attempt = 0; attempt <= RetryMax; attempt++)
{
var attemptSw = Stopwatch.StartNew();
try
{
using var response = await client.PostAsJsonAsync(endpoint, request);
var json = await response.Content.ReadAsStringAsync();
attemptSw.Stop();
attemptTimes.Add(attemptSw.ElapsedMilliseconds);
if (response.IsSuccessStatusCode)
{
using var document = JsonDocument.Parse(json);
var root = document.RootElement;
var result = root.GetProperty("result");
var probability = result
.GetProperty("answers")
.GetProperty("xss")
.GetProperty("noul")
.GetDouble();
int? inputTokens = null;
if (result.TryGetProperty("usage", out var usage) &&
usage.TryGetProperty("input_tokens", out var tokenElement) &&
tokenElement.ValueKind == JsonValueKind.Number)
{
inputTokens = tokenElement.GetInt32();
}
if (!double.IsFinite(probability) || probability < 0 || probability > 1)
{
retryLog.Add($"{model}: invalid probability");
return new CallResult
{
Probability = null,
Ms = attemptTimes.Sum(),
Tokens = null,
Status = "Invalid Probability",
RetryCount = retryCount,
RetryWaitMs = retryWaitMs,
};
}
return new CallResult
{
Probability = probability,
Ms = attemptTimes.Sum(),
Tokens = inputTokens,
Status = "OK",
RetryCount = retryCount,
RetryWaitMs = retryWaitMs,
};
}
int code = (int)response.StatusCode;
details.Add($"HTTP {code}");
retryLog.Add($"{model}: HTTP {code}");
if ((code == 429 || code >= 500) && attempt < RetryMax)
{
retryCount++;
long wait = 1000L * (1L << attempt);
retryWaitMs += wait;
details.Add($"Retry wait {wait} ms");
await Task.Delay((int)wait);
continue;
}
return new CallResult
{
Probability = null,
Ms = attemptTimes.Sum(),
Tokens = null,
Status = $"HTTP {code}",
RetryCount = retryCount,
RetryWaitMs = retryWaitMs,
};
}
catch (Exception ex) when (ex is HttpRequestException || ex is TaskCanceledException)
{
attemptSw.Stop();
attemptTimes.Add(attemptSw.ElapsedMilliseconds);
details.Add($"{ex.GetType().Name}: {ex.Message}");
retryLog.Add($"{model}: {ex.GetType().Name}");
if (attempt < RetryMax)
{
retryCount++;
long wait = 1000L * (1L << attempt);
retryWaitMs += wait;
details.Add($"Retry wait {wait} ms");
await Task.Delay((int)wait);
continue;
}
return new CallResult
{
Probability = null,
Ms = attemptTimes.Sum(),
Tokens = null,
Status = "Error",
RetryCount = retryCount,
RetryWaitMs = retryWaitMs,
};
}
catch (Exception ex)
{
attemptSw.Stop();
attemptTimes.Add(attemptSw.ElapsedMilliseconds);
details.Add($"{ex.GetType().Name}: {ex.Message}");
retryLog.Add($"{model}: {ex.GetType().Name}");
return new CallResult
{
Probability = null,
Ms = attemptTimes.Sum(),
Tokens = null,
Status = "Error",
RetryCount = retryCount,
RetryWaitMs = retryWaitMs,
};
}
}
return new CallResult
{
Probability = null,
Ms = attemptTimes.Sum(),
Tokens = null,
Status = "Error",
RetryCount = retryCount,
RetryWaitMs = retryWaitMs,
};
}
private static List<string> LoadPayloads()
{
var candidates = new[]
{
Path.Combine(Directory.GetCurrentDirectory(), "xss-benchmark-100.txt"),
Path.Combine(AppContext.BaseDirectory, "xss-benchmark-100.txt"),
Path.Combine(Directory.GetCurrentDirectory(), "..", "..", "..", "xss-benchmark-100.txt"),
"xss-benchmark-100.txt"
};
foreach (var c in candidates)
{
try
{
if (File.Exists(c))
{
var lines = File.ReadAllLines(c, Encoding.UTF8)
.Select(l => l.TrimEnd('\r'))
.Where(l => !string.IsNullOrWhiteSpace(l))
.ToList();
if (lines.Count > 0) return lines;
}
}
catch { }
}
return new List<string>();
}
private struct CallResult
{
public double? Probability;
public long Ms;
public int? Tokens;
public string Status;
public int RetryCount;
public long RetryWaitMs;
}
private class ResultRow
{
public int Index;
public string Payload;
public double? Probability;
public long Ms;
public int? Tokens;
public string Status;
public int RetryCount;
public long RetryWaitMs;
public ResultRow(int index, string payload, double? probability, long ms, int? tokens, string status, int retryCount, long retryWaitMs)
{
Index = index; Payload = payload; Probability = probability; Ms = ms; Tokens = tokens; Status = status;
RetryCount = retryCount; RetryWaitMs = retryWaitMs;
}
}
private class LowSample
{
public int Index;
public string Payload;
public double? ClefProb;
public double? FlashProb;
public LowSample(int index, string payload, double? clefProb, double? flashProb)
{
Index = index; Payload = payload; ClefProb = clefProb; FlashProb = flashProb;
}
}
private class ModelMetrics
{
public int RequestCount;
public int SuccessCount;
public int FailCount;
public int? TotalTokens;
public long ApiTotalMs;
public double? AvgLatency;
public long? MinLatency;
public long? MaxLatency;
public double? P50;
public double? P95;
public double? P99;
public int RetryCount;
public long RetryWaitMs;
public int SleepMs;
public double? AvgProb;
public int LowCount;
}
private class RunRecord
{
public int Run;
public long WallMs;
public long ClefApiMs;
public long FlashApiMs;
public Dictionary<string, List<ResultRow>> Results;
public Dictionary<string, int> RetryCount;
public Dictionary<string, long> RetryWaitMs;
public List<LowSample> LowSamples;
public RunRecord(int run, long wallMs, long clefApiMs, long flashApiMs,
Dictionary<string, List<ResultRow>> results,
Dictionary<string, int> retryCount,
Dictionary<string, long> retryWaitMs,
List<LowSample> lowSamples)
{
Run = run; WallMs = wallMs; ClefApiMs = clefApiMs; FlashApiMs = flashApiMs;
Results = results; RetryCount = retryCount; RetryWaitMs = retryWaitMs; LowSamples = lowSamples;
}
}
private class TeeTextWriter : TextWriter
{
private readonly TextWriter _a;
private readonly TextWriter _b;
public TeeTextWriter(TextWriter a, TextWriter b) { _a = a; _b = b; }
public override Encoding Encoding => _a.Encoding;
public override void Write(char value) { _a.Write(value); _b.Write(value); }
public override void Write(string? value) { _a.Write(value); _b.Write(value); }
public override void Write(char[] buffer, int index, int count) { _a.Write(buffer, index, count); _b.Write(buffer, index, count); }
public override void WriteLine() { _a.WriteLine(); _b.WriteLine(); }
public override void WriteLine(string? value) { _a.WriteLine(value); _b.WriteLine(value); }
public override void Flush() { _a.Flush(); _b.Flush(); }
}
}xss-benchmark-100.txt
100 筆測試字串,每行一筆。這些是拿來分類的輸入,範例不應該直接插入正式網站執行
<script>alert(1)</script>
<script>console.log(1)</script>
<script>confirm(1)</script>
<script>prompt(1)</script>
<ScRiPt>alert(1)</ScRiPt>
<SCRIPT>alert(1)</SCRIPT>
<script type="text/javascript">alert(1)</script>
<script type=module>alert(1)</script>
<script>window.alert(1)</script>
<script>globalThis.alert(1)</script>
<script>setTimeout(()=>alert(1),0)</script>
<script>setInterval(()=>alert(1),1000)</script>
<script>eval("alert(1)")</script>
<script>Function("alert(1)")()</script>
<script>document.body.dataset.xss="1"</script>
<script>location.hash="#xss-test"</script>
<script>document.title="xss-test"</script>
<script>console.warn("xss-test")</script>
<script>console.info("xss-test")</script>
<script>void alert(1)</script>
<img src=x onerror=alert(1)>
<img src=x onerror="console.log(1)">
<img src=x onerror="confirm(1)">
<img src=x onerror="prompt(1)">
<img src=x onerror="window.alert(1)">
<img src=x onerror="document.title='xss-test'">
<body onload=alert(1)>
<input autofocus onfocus=alert(1)>
<textarea autofocus onfocus=alert(1)></textarea>
<select autofocus onfocus=alert(1)></select>
<details open ontoggle=alert(1)>x</details>
<video src=x onerror=alert(1)></video>
<audio src=x onerror=alert(1)></audio>
<source src=x onerror=alert(1)>
<object data=x onerror=alert(1)></object>
<iframe onload=alert(1)></iframe>
<marquee onstart=alert(1)>x</marquee>
<button onclick=alert(1)>click</button>
<div onmouseover=alert(1)>hover</div>
<input onchange=alert(1) value=x>
<svg onload=alert(1)></svg>
<svg onload="console.log(1)"></svg>
<svg><script>alert(1)</script></svg>
<svg><script>confirm(1)</script></svg>
<svg><script>prompt(1)</script></svg>
<svg><a href="javascript:alert(1)">click</a></svg>
<svg><circle onclick="alert(1)" r="5"/></svg>
<svg><rect onmouseover="alert(1)" width="10" height="10"/></svg>
<svg><text onclick="alert(1)">click</text></svg>
<svg><animate onbegin="alert(1)" attributeName="x" dur="1s"/></svg>
<math><mtext><script>alert(1)</script></mtext></math>
<math><mtext><img src=x onerror=alert(1)></mtext></math>
<svg><foreignObject><body xmlns="http://www.w3.org/1999/xhtml" onload="alert(1)"></body></foreignObject></svg>
<svg><foreignObject><img xmlns="http://www.w3.org/1999/xhtml" src=x onerror=alert(1) /></foreignObject></svg>
<svg><g onmouseover="alert(1)"><rect width="10" height="10" /></g></svg>
<a href="javascript:alert(1)">click</a>
<a href="javascript:confirm(1)">click</a>
<a href="javascript:prompt(1)">click</a>
<a href="JaVaScRiPt:alert(1)">click</a>
<a href="javascript:window.alert(1)">click</a>
<a href="javascript:void(alert(1))">click</a>
<iframe src="javascript:alert(1)"></iframe>
<form action="javascript:alert(1)"><button>submit</button></form>
<button formaction="javascript:alert(1)">submit</button>
javascript:alert(1)
javascript:confirm(1)
javascript:prompt(1)
javascript:console.log(1)
data:text/html,<script>alert(1)</script>
data:text/html,<svg onload=alert(1)></svg>
" onmouseover="alert(1)
" onfocus="alert(1)" autofocus="
" onclick="alert(1)
" onerror="alert(1)
"><img src=x onerror=alert(1)>
"><svg onload=alert(1)>
"><script>alert(1)</script>
"><iframe onload=alert(1)></iframe>
"><details open ontoggle=alert(1)>
"/><img src=x onerror=alert(1)>
';alert(1);//
";alert(1);//
';console.log(1);//
";console.log(1);//
';confirm(1);//
";prompt(1);//
</script><script>alert(1)</script>
</script><img src=x onerror=alert(1)>
</script><svg onload=alert(1)>
${alert(1)}
<script>alert(1)</script>
<script>alert(1)</script>
<script>alert(1)</script>
%3Cscript%3Ealert(1)%3C%2Fscript%3E
%3Cimg%20src%3Dx%20onerror%3Dalert(1)%3E
<img src=x onerror=alert(1)>
javascript:alert(1)
javascript:alert(1)
javascript:alert(1)
<script>alert(1)</script>output.txt
下面放最終摘要;完整五輪每個低分 Payload 的明細也都在 output.txt 裡。
載入 100 筆 XSS 測試樣本
測試模型:clef, clef-flash
低分門檻:< 90%
執行次數:5
Warm-up 次數:每個模型 5 次(不納入統計)
開始 Warm-up...
clef Warm-up 完成 (5 次)
clef-flash Warm-up 完成 (5 次)
Warm-up 結束,正式測試開始。
######################################################################
第 1 / 5 輪 Benchmark
######################################################################
模型 clef clef-flash
----------------------------------------------------------
Request 數量 100 100
成功數量 100 100
失敗數量 0 0
Input Tokens 20,604 20,604
API 總耗時 (ms) 40,180 36,356
API 總耗時 (秒) 40.18 36.36
平均 API 延遲 (ms) 401.80 363.56
P50 (ms) 344.00 277.50
P95 (ms) 780.60 1153.80
P99 (ms) 940.16 1278.66
最短 API 延遲 (ms) 169 113
最長 API 延遲 (ms) 956 1344
Retry 次數 0 0
Retry 等待時間 (ms) 0 0
固定 Sleep 時間 (ms) 20,000 20,000
XSS 平均機率 (%) 97.91 91.07
低於 90% 數量 1 16
預估費用 (USD) US$0.004945 US$0.000783
本輪 API 耗時比較:
Clef API 總耗時:40180 ms (40.18 s)
Clef Flash API 總耗時:36356 ms (36.36 s)
較快模型:Clef Flash
總耗時差距:3824 ms (3.82 s)
速度倍率:1.11 x
注意:API 總耗時不包含任何 Sleep、Task.Delay、
Retry Backoff 或 Warm-up。
低分 Payload 明細(< 90%,共 17 筆):
# Clef Flash Diff Payload
------------------------------------------------------------------------------------------
8 98.73% 78.63% 20.10 pp <script type="module">alert(1)</script>
14 98.98% 87.64% 11.34 pp <script>Function("alert(1)")()</script>
15 99.09% 86.54% 12.55 pp <script>document.body.dataset.xss="1"</script>
31 98.62% 60.42% 38.20 pp <details open="" ontoggle="alert(1)">x</details>
33 98.95% 88.75% 10.20 pp <audio src="x" onerror="alert(1)"></audio>
50 97.91% 30.34% 67.57 pp <svg><animate onbegin="alert(1)" attributeName="x" dur="1s"></animate></svg>
53 98.30% 89.82% 8.48 pp <svg><foreignObject></foreignObject></svg>
54 98.16% 88.17% 9.99 pp <svg><foreignObject><img xmlns="http://www.w3.org/1999/xhtml" src="x" onerror="alert(1)"></foreignObject></svg>
79 98.08% 67.07% 31.01 pp "></code><details open="" ontoggle="alert(1)"><code lang="text">
83 93.43% 74.71% 18.72 pp ';console.log(1);//
84 93.43% 65.29% 28.14 pp ";console.log(1);//
86 95.98% 73.97% 22.01 pp ";prompt(1);//
90 98.06% 17.72% 80.34 pp ${alert(1)}
91 69.25% 95.57% -26.32 pp <script>alert(1)</script>
94 98.09% 86.53% 11.56 pp %3Cscript%3Ealert(1)%3C%2Fscript%3E
97 96.57% 83.67% 12.90 pp javascript:alert(1)
99 95.22% 84.06% 11.16 pp javascript:alert(1)
百分位數計算方式:成功 Request 的耗時排序後,採線性插值法 (P50/P95/P99)。
API 總耗時 = 所有 HTTP 嘗試(含失敗)的端到端耗時加總,不含 Sleep/Backoff/Warm-up。
######################################################################
第 2 / 5 輪 Benchmark
######################################################################
模型 clef clef-flash
----------------------------------------------------------
Request 數量 100 100
成功數量 100 100
失敗數量 0 0
Input Tokens 20,604 20,604
API 總耗時 (ms) 51,334 27,635
API 總耗時 (秒) 51.33 27.64
平均 API 延遲 (ms) 513.34 276.35
P50 (ms) 385.00 140.00
P95 (ms) 1058.35 959.25
P99 (ms) 1268.15 1196.90
最短 API 延遲 (ms) 167 114
最長 API 延遲 (ms) 1382 1286
Retry 次數 0 0
Retry 等待時間 (ms) 0 0
固定 Sleep 時間 (ms) 20,000 20,000
XSS 平均機率 (%) 97.91 91.07
低於 90% 數量 1 16
預估費用 (USD) US$0.004945 US$0.000783
本輪 API 耗時比較:
Clef API 總耗時:51334 ms (51.33 s)
Clef Flash API 總耗時:27635 ms (27.64 s)
較快模型:Clef Flash
總耗時差距:23699 ms (23.70 s)
速度倍率:1.86 x
注意:API 總耗時不包含任何 Sleep、Task.Delay、
Retry Backoff 或 Warm-up。
低分 Payload 明細(< 90%,共 17 筆):
# Clef Flash Diff Payload
------------------------------------------------------------------------------------------
8 98.73% 78.63% 20.10 pp <script type="module">alert(1)</script>
14 98.98% 87.64% 11.34 pp <script>Function("alert(1)")()</script>
15 99.09% 86.54% 12.55 pp <script>document.body.dataset.xss="1"</script>
31 98.62% 60.42% 38.20 pp <details open="" ontoggle="alert(1)">x</details>
33 98.95% 88.75% 10.20 pp <audio src="x" onerror="alert(1)"></audio>
50 97.91% 30.34% 67.57 pp <svg><animate onbegin="alert(1)" attributeName="x" dur="1s"></animate></svg>
53 98.30% 89.82% 8.48 pp <svg><foreignObject></foreignObject></svg>
54 98.16% 88.17% 9.99 pp <svg><foreignObject><img xmlns="http://www.w3.org/1999/xhtml" src="x" onerror="alert(1)"></foreignObject></svg>
79 98.08% 67.07% 31.01 pp "></code><details open="" ontoggle="alert(1)"><code lang="text">
83 93.43% 74.71% 18.72 pp ';console.log(1);//
84 93.43% 65.29% 28.14 pp ";console.log(1);//
86 95.98% 73.97% 22.01 pp ";prompt(1);//
90 98.06% 17.72% 80.34 pp ${alert(1)}
91 69.25% 95.57% -26.32 pp <script>alert(1)</script>
94 98.09% 86.53% 11.56 pp %3Cscript%3Ealert(1)%3C%2Fscript%3E
97 96.57% 83.67% 12.90 pp javascript:alert(1)
99 95.22% 84.06% 11.16 pp javascript:alert(1)
百分位數計算方式:成功 Request 的耗時排序後,採線性插值法 (P50/P95/P99)。
API 總耗時 = 所有 HTTP 嘗試(含失敗)的端到端耗時加總,不含 Sleep/Backoff/Warm-up。
######################################################################
第 3 / 5 輪 Benchmark
######################################################################
模型 clef clef-flash
----------------------------------------------------------
Request 數量 100 100
成功數量 100 100
失敗數量 0 0
Input Tokens 20,604 20,604
API 總耗時 (ms) 42,021 27,694
API 總耗時 (秒) 42.02 27.69
平均 API 延遲 (ms) 420.21 276.94
P50 (ms) 347.00 145.00
P95 (ms) 754.00 754.50
P99 (ms) 996.24 1139.60
最短 API 延遲 (ms) 155 112
最長 API 延遲 (ms) 1317 1694
Retry 次數 0 0
Retry 等待時間 (ms) 0 0
固定 Sleep 時間 (ms) 20,000 20,000
XSS 平均機率 (%) 97.91 91.07
低於 90% 數量 1 16
預估費用 (USD) US$0.004945 US$0.000783
本輪 API 耗時比較:
Clef API 總耗時:42021 ms (42.02 s)
Clef Flash API 總耗時:27694 ms (27.69 s)
較快模型:Clef Flash
總耗時差距:14327 ms (14.33 s)
速度倍率:1.52 x
注意:API 總耗時不包含任何 Sleep、Task.Delay、
Retry Backoff 或 Warm-up。
低分 Payload 明細(< 90%,共 17 筆):
# Clef Flash Diff Payload
------------------------------------------------------------------------------------------
8 98.73% 78.63% 20.10 pp <script type="module">alert(1)</script>
14 98.98% 87.64% 11.34 pp <script>Function("alert(1)")()</script>
15 99.09% 86.54% 12.55 pp <script>document.body.dataset.xss="1"</script>
31 98.62% 60.42% 38.20 pp <details open="" ontoggle="alert(1)">x</details>
33 98.95% 88.75% 10.20 pp <audio src="x" onerror="alert(1)"></audio>
50 97.91% 30.34% 67.57 pp <svg><animate onbegin="alert(1)" attributeName="x" dur="1s"></animate></svg>
53 98.30% 89.82% 8.48 pp <svg><foreignObject></foreignObject></svg>
54 98.16% 88.17% 9.99 pp <svg><foreignObject><img xmlns="http://www.w3.org/1999/xhtml" src="x" onerror="alert(1)"></foreignObject></svg>
79 98.08% 67.07% 31.01 pp "></code><details open="" ontoggle="alert(1)"><code lang="text">
83 93.43% 74.71% 18.72 pp ';console.log(1);//
84 93.43% 65.29% 28.14 pp ";console.log(1);//
86 95.98% 73.97% 22.01 pp ";prompt(1);//
90 98.06% 17.72% 80.34 pp ${alert(1)}
91 69.25% 95.57% -26.32 pp <script>alert(1)</script>
94 98.09% 86.53% 11.56 pp %3Cscript%3Ealert(1)%3C%2Fscript%3E
97 96.57% 83.67% 12.90 pp javascript:alert(1)
99 95.22% 84.06% 11.16 pp javascript:alert(1)
百分位數計算方式:成功 Request 的耗時排序後,採線性插值法 (P50/P95/P99)。
API 總耗時 = 所有 HTTP 嘗試(含失敗)的端到端耗時加總,不含 Sleep/Backoff/Warm-up。
######################################################################
第 4 / 5 輪 Benchmark
######################################################################
模型 clef clef-flash
----------------------------------------------------------
Request 數量 100 100
成功數量 100 100
失敗數量 0 0
Input Tokens 20,604 20,604
API 總耗時 (ms) 45,645 22,464
API 總耗時 (秒) 45.65 22.46
平均 API 延遲 (ms) 456.45 224.64
P50 (ms) 342.00 138.50
P95 (ms) 873.15 518.25
P99 (ms) 1188.35 1100.01
最短 API 延遲 (ms) 133 110
最長 API 延遲 (ms) 4490 1101
Retry 次數 0 0
Retry 等待時間 (ms) 0 0
固定 Sleep 時間 (ms) 20,000 20,000
XSS 平均機率 (%) 97.91 91.07
低於 90% 數量 1 16
預估費用 (USD) US$0.004945 US$0.000783
本輪 API 耗時比較:
Clef API 總耗時:45645 ms (45.65 s)
Clef Flash API 總耗時:22464 ms (22.46 s)
較快模型:Clef Flash
總耗時差距:23181 ms (23.18 s)
速度倍率:2.03 x
注意:API 總耗時不包含任何 Sleep、Task.Delay、
Retry Backoff 或 Warm-up。
低分 Payload 明細(< 90%,共 17 筆):
# Clef Flash Diff Payload
------------------------------------------------------------------------------------------
8 98.73% 78.63% 20.10 pp <script type="module">alert(1)</script>
14 98.98% 87.64% 11.34 pp <script>Function("alert(1)")()</script>
15 99.09% 86.54% 12.55 pp <script>document.body.dataset.xss="1"</script>
31 98.62% 60.42% 38.20 pp <details open="" ontoggle="alert(1)">x</details>
33 98.95% 88.75% 10.20 pp <audio src="x" onerror="alert(1)"></audio>
50 97.91% 30.34% 67.57 pp <svg><animate onbegin="alert(1)" attributeName="x" dur="1s"></animate></svg>
53 98.30% 89.82% 8.48 pp <svg><foreignObject></foreignObject></svg>
54 98.16% 88.17% 9.99 pp <svg><foreignObject><img xmlns="http://www.w3.org/1999/xhtml" src="x" onerror="alert(1)"></foreignObject></svg>
79 98.08% 67.07% 31.01 pp "></code><details open="" ontoggle="alert(1)"><code lang="text">
83 93.43% 74.71% 18.72 pp ';console.log(1);//
84 93.43% 65.29% 28.14 pp ";console.log(1);//
86 95.98% 73.97% 22.01 pp ";prompt(1);//
90 98.06% 17.72% 80.34 pp ${alert(1)}
91 69.25% 95.57% -26.32 pp <script>alert(1)</script>
94 98.09% 86.53% 11.56 pp %3Cscript%3Ealert(1)%3C%2Fscript%3E
97 96.57% 83.67% 12.90 pp javascript:alert(1)
99 95.22% 84.06% 11.16 pp javascript:alert(1)
百分位數計算方式:成功 Request 的耗時排序後,採線性插值法 (P50/P95/P99)。
API 總耗時 = 所有 HTTP 嘗試(含失敗)的端到端耗時加總,不含 Sleep/Backoff/Warm-up。
######################################################################
第 5 / 5 輪 Benchmark
######################################################################
模型 clef clef-flash
----------------------------------------------------------
Request 數量 100 100
成功數量 100 100
失敗數量 0 0
Input Tokens 20,604 20,604
API 總耗時 (ms) 40,995 33,443
API 總耗時 (秒) 40.99 33.44
平均 API 延遲 (ms) 409.95 334.43
P50 (ms) 337.00 155.50
P95 (ms) 769.85 916.55
P99 (ms) 967.87 1166.09
最短 API 延遲 (ms) 134 115
最長 API 延遲 (ms) 1648 1175
Retry 次數 0 0
Retry 等待時間 (ms) 0 0
固定 Sleep 時間 (ms) 20,000 20,000
XSS 平均機率 (%) 97.91 91.07
低於 90% 數量 1 16
預估費用 (USD) US$0.004945 US$0.000783
本輪 API 耗時比較:
Clef API 總耗時:40995 ms (40.99 s)
Clef Flash API 總耗時:33443 ms (33.44 s)
較快模型:Clef Flash
總耗時差距:7552 ms (7.55 s)
速度倍率:1.23 x
注意:API 總耗時不包含任何 Sleep、Task.Delay、
Retry Backoff 或 Warm-up。
低分 Payload 明細(< 90%,共 17 筆):
# Clef Flash Diff Payload
------------------------------------------------------------------------------------------
8 98.73% 78.63% 20.10 pp <script type="module">alert(1)</script>
14 98.98% 87.64% 11.34 pp <script>Function("alert(1)")()</script>
15 99.09% 86.54% 12.55 pp <script>document.body.dataset.xss="1"</script>
31 98.62% 60.42% 38.20 pp <details open="" ontoggle="alert(1)">x</details>
33 98.95% 88.75% 10.20 pp <audio src="x" onerror="alert(1)"></audio>
50 97.91% 30.34% 67.57 pp <svg><animate onbegin="alert(1)" attributeName="x" dur="1s"></animate></svg>
53 98.30% 89.82% 8.48 pp <svg><foreignObject></foreignObject></svg>
54 98.16% 88.17% 9.99 pp <svg><foreignObject><img xmlns="http://www.w3.org/1999/xhtml" src="x" onerror="alert(1)"></foreignObject></svg>
79 98.08% 67.07% 31.01 pp "></code><details open="" ontoggle="alert(1)"><code lang="text">
83 93.43% 74.71% 18.72 pp ';console.log(1);//
84 93.43% 65.29% 28.14 pp ";console.log(1);//
86 95.98% 73.97% 22.01 pp ";prompt(1);//
90 98.06% 17.72% 80.34 pp ${alert(1)}
91 69.25% 95.57% -26.32 pp <script>alert(1)</script>
94 98.09% 86.53% 11.56 pp %3Cscript%3Ealert(1)%3C%2Fscript%3E
97 96.57% 83.67% 12.90 pp javascript:alert(1)
99 95.22% 84.06% 11.16 pp javascript:alert(1)
百分位數計算方式:成功 Request 的耗時排序後,採線性插值法 (P50/P95/P99)。
API 總耗時 = 所有 HTTP 嘗試(含失敗)的端到端耗時加總,不含 Sleep/Backoff/Warm-up。
======================================================================
CLEF VS CLEF FLASH - FINAL BENCHMARK
======================================================================
輪次 Clef API 總秒數 Flash API 總秒數 較快模型
----------------------------------------------------------
Run 1 40.18 36.36 Clef Flash
Run 2 51.33 27.64 Clef Flash
Run 3 42.02 27.69 Clef Flash
Run 4 45.65 22.46 Clef Flash
Run 5 40.99 33.44 Clef Flash
----------------------------------------------------------
五輪合計 220.18 147.59
五輪總 API 耗時差距:72583 ms (72.58 秒)
五輪總 API 速度倍率:1.49 x(Clef Flash 較快)
Clef 勝出輪數:0,Clef Flash 勝出輪數:5
Clef 總 Input Tokens:103,020
Clef Flash 總 Input Tokens:103,020
Clef 總預估成本:US$0.024725
Clef Flash 總預估成本:US$0.003915
成本節省比例:84.17%
平均每 Request Input Tokens:Clef 206.04,Clef Flash 206.04
每 100 萬次相同規模請求預估成本:Clef US$49.45,Clef Flash US$7.83
======================================================================
五輪彙總
======================================================================
clef: OK=500/500, 低分=5, 平均機率=97.91%, 平均延遲=440.35 ms, P50=344.50, P95=889.15, P99=1203.64, 重試=0, 重試等待=0 ms
clef-flash: OK=500/500, 低分=80, 平均機率=91.07%, 平均延遲=295.18 ms, P50=148.00, P95=1082.80, P99=1196.63, 重試=0, 重試等待=0 ms
整體牆鐘時間:576030 ms (576.03 s)
======================================================================
========== Retry 詳細記錄 ==========
======================================================================
(無 Retry)Flash 在 API 平均延遲與成本上有絕對優勢(快 1.49 倍、費用省 84%);但在判斷機率上,Clef (27B) 對於特定語境(如 SVG、HTML Event Handler 與 Template Literal)的把握度明顯更高(低於 90% 的次數僅 5 次 vs Flash 的 80 次),但是因為兩個參數有時候判斷相異性過大,可能要好好調整 Prompt 或是 自己微調自己的需求其實 Flash也是很好的 ,因為夠便宜 :)
Cloudflare 官方模型資料:Clef、Clef Flash
---
Before you read this, the article was in a superposition of human and AI. Upon observation, the wave function collapsed into a sleep-deprived developer.
