/
Beckshome.com: Thomas Beck's Blog
Copyright © 2023
2023-07-22T22:15:40Z
/beckshome.jpeg
/2022/11/lucene-blazor-part-5-highlighting
Lucene + Blazor, Part 5: Highlighting
2022-11-25T00:00:00Z
<p>In this final installment of my Blazor + Lucene.Net series, we'll be adding highlights for the search terms found in the header and body text of each of our results. The implementation of highlighting makes use of the Lucene.Net.Highlighter library, plugging this library into a simple method that can be used as a filter for search results to highlight key terms.</p>
<p>The code and code narrative below reflects the changes that have been made on top of the first 4 posts. All <a href="https://github.com/thbst16/dotnet-lucene-search/tree/main/5-Highlighting">source code is available online</a> for this highlighting post.</p>
<p><strong>Sample App</strong></p>
<p>The sample application lets you search over 3,000 waffle text entries, returning paginated search results. Auto-complete functionality provides suggestion for the most relevant search terms in the waffle text index. On top of the search results, two attributes (Scholars and Universities) are available as facets. Finally, search results in the header and body of the waffle text are highlighted. The site is available online at <a href="https://dotnet-lucene-search.azurewebsites.net/">https://dotnet-lucene-search.azurewebsites.net/</a></p>
<p><img src="https://s3.amazonaws.com/s3.beckshome.com/20221122-dotnet-lucene-highlighting.jpeg" alt="Highlighting" /></p>
<p><strong>Highlighted Search Terms</strong></p>
<p>There are two modifications to the search engine (SearchEngine.cs) to enable search highlighting:</p>
<ol>
<li>A new static method is added (GenerateHighlightedText()), which takes in the components needed by the Lucene.Net.Highlighter library's GetBestFragments() method to surround keywords with the HTML Mark and Strong tags for highlighting.</li>
<li>The existing FacetedSearch method is modified to pass the WaffleHead and WaffleBody text through the GenerateHighlightedText() method to highlight text in these two fields. Lines 135 and 137 below.</li>
</ol>
<pre data-enlighter-language="csharp">
using Bogus;
using Lucene.Net.Analysis;
using Lucene.Net.Analysis.Standard;
using Lucene.Net.QueryParsers.Classic;
using Lucene.Net.Documents;
using Lucene.Net.Facet;
using Lucene.Net.Facet.Taxonomy;
using Lucene.Net.Facet.Taxonomy.Directory;
using Lucene.Net.Index;
using Lucene.Net.Search;
using Lucene.Net.Search.Highlight;
using Lucene.Net.Search.Spell;
using Lucene.Net.Search.Suggest.Analyzing;
using Lucene.Net.Store;
using Lucene.Net.Util;
using System.Text.RegularExpressions;
namespace search.Shared
{
public class SearchEngine{
public static List<WaffleText> Data {get; set;}
private static RAMDirectory _indexDirectory;
private static RAMDirectory _facetDirectory;
public static IndexWriter indexWriter { get; set; }
public static DirectoryTaxonomyWriter taxoWriter { get; set; }
private static FacetsConfig facetConfig = new FacetsConfig();
public static void GetData(int Rand, int WaffleCount)
{
Randomizer.Seed = new Random(Rand);
var testWaffles = new Faker<WaffleText>()
.RuleFor(wt => wt.GUID, f => Guid.NewGuid().ToString())
.RuleFor(
property: wt => wt.WaffleHead,
setter: (f, wt) => f.WaffleTitle())
.RuleFor(
property: wt => wt.WaffleBody,
setter: (f, wt) => f.WaffleText(
paragraphs: 2,
includeHeading: false))
.RuleFor(wt => wt.WaffleScholar, f => f.PickRandom<WaffleScholar>())
.RuleFor(wt => wt.WaffleUniversity, f => f.PickRandom<WaffleUniversity>());
var waffles = testWaffles.Generate(WaffleCount);
Data = new List<WaffleText>();
foreach(WaffleText wt in waffles)
{
Data.Add(wt);
}
}
public static void Index()
{
const LuceneVersion lv = LuceneVersion.LUCENE_48;
Analyzer a = new StandardAnalyzer(lv);
_indexDirectory = new RAMDirectory();
_facetDirectory = new RAMDirectory();
var config = new IndexWriterConfig(lv, a);
indexWriter = new IndexWriter(_indexDirectory, config);
taxoWriter = new DirectoryTaxonomyWriter(_facetDirectory);
var doc = new Document();
foreach (WaffleText wt in Data)
{
doc = new Document();
doc.Add(new StringField("GUID", wt.GUID, Field.Store.YES));
doc.Add(new TextField("WaffleHead", wt.WaffleHead, Field.Store.YES));
doc.Add(new TextField("WaffleBody", wt.WaffleBody, Field.Store.YES));
doc.Add(new TextField("HeadBody", wt.WaffleHead + " " + wt.WaffleBody, Field.Store.YES));
doc.Add(new TextField("WaffleScholarTxt", wt.WaffleScholar.ToString(), Field.Store.YES));
doc.Add(new TextField("WaffleUniversityTxt", wt.WaffleUniversity.ToString(), Field.Store.YES));
doc.Add(new FacetField("WaffleScholar", wt.WaffleScholar.ToString()));
doc.Add(new FacetField("WaffleUniversity", wt.WaffleUniversity.ToString()));
indexWriter.AddDocument(facetConfig.Build(taxoWriter, doc));
}
indexWriter.Commit();
taxoWriter.Commit();
}
public static void Dispose()
{
indexWriter.Dispose();
taxoWriter.Dispose();
_indexDirectory.Dispose();
_facetDirectory.Dispose();
}
public static SearchModel FacetedSearch(string input, int page, List<string> scholarDrillDowns = null, List<string> universityDrillDowns = null)
{
const LuceneVersion lv = LuceneVersion.LUCENE_48;
Analyzer a = new StandardAnalyzer(lv);
string[] fnames = { "GUID", "WaffleHead", "WaffleBody" };
var multiFieldQP = new MultiFieldQueryParser(lv, fnames, a);
string _input = EscapeSearchTerm(input.Trim());
Query query = multiFieldQP.Parse(_input);
// Add drill down query
DrillDownQuery ddq = new DrillDownQuery(facetConfig, query);
if (scholarDrillDowns is not null)
{
foreach (string scholar in scholarDrillDowns)
{
ddq.Add("WaffleScholar", scholar);
}
}
if (universityDrillDowns is not null)
{
foreach (string university in universityDrillDowns)
{
ddq.Add("WaffleUniversity", university);
}
}
using (DirectoryReader indexReader = DirectoryReader.Open(_indexDirectory))
using (TaxonomyReader taxoReader = new DirectoryTaxonomyReader(_facetDirectory))
{
IndexSearcher searcher = new IndexSearcher(indexReader);
// Execute document search and return collection of WaffleText class
ScoreDoc[] docs = searcher.Search(ddq, null, 1000).ScoreDocs;
var waffles = new List<WaffleText>();
int first = (page-1)*5;
int last = first + 5;
for (int i = first; i < last && i < docs.Length; i++)
{
Document doc = searcher.Doc(docs[i].Doc);
WaffleText _waffle = new WaffleText();
_waffle.GUID = doc.Get("GUID");
_waffle.WaffleHead = GenerateHighlightedText(a, query, doc.Get("WaffleHead"), "WaffleHead");
if (_waffle.WaffleHead == string.Empty) {_waffle.WaffleHead = doc.Get("WaffleHead");}
_waffle.WaffleBody = GenerateHighlightedText(a, query, doc.Get("WaffleBody"), "WaffleBody");
if (_waffle.WaffleBody == string.Empty) {_waffle.WaffleBody = doc.Get("WaffleBody");}
_waffle.WaffleScholar = (WaffleScholar)Enum.Parse(typeof(WaffleScholar), doc.Get("WaffleScholarTxt"));
_waffle.WaffleUniversity = (WaffleUniversity)Enum.Parse(typeof(WaffleUniversity), doc.Get("WaffleUniversityTxt"));
waffles.Add(_waffle);
}
var returnModel = new SearchModel();
returnModel.CurrentPageSearchResults = waffles;
returnModel.SearchText = _input;
returnModel.ResultsCount = docs.Length;
returnModel.PageCount = (int)Math.Ceiling(docs.Length/5.0);
returnModel.CurrentPage = page;
// Execute facets search and return collection of FacetResults class
FacetsCollector fc = new FacetsCollector();
FacetsCollector.Search(searcher, ddq, 100, fc);
IList<FacetResult> results = new List<FacetResult>();
Facets facets = new FastTaxonomyFacetCounts(taxoReader, facetConfig, fc);
results.Add(facets.GetTopChildren(100, "WaffleScholar"));
results.Add(facets.GetTopChildren(100, "WaffleUniversity"));
returnModel.FacetResults = results;
return returnModel;
}
}
public static List<string> SearchAhead(string input)
{
const LuceneVersion lv = LuceneVersion.LUCENE_48;
Analyzer a = new StandardAnalyzer(lv);
var dirReader = DirectoryReader.Open(_indexDirectory);
LuceneDictionary dictionary = new LuceneDictionary(dirReader, "HeadBody");
RAMDirectory _d = new RAMDirectory();
AnalyzingInfixSuggester analyzingSuggester = new AnalyzingInfixSuggester(lv, _d, a);
analyzingSuggester.Build(dictionary);
var lookupResultList = analyzingSuggester.DoLookup(input.Trim(), false, 9);
List<string> returnModel = new List<string>();
foreach(var result in lookupResultList)
{
returnModel.Add(result.Key);
}
return returnModel;
dirReader.Dispose();
}
// Lucene supports escapting the following chars: + - && || ! ( ) { } [ ] ^ " ~ * ? : \
// To make it easier, I remove / replace the text altogether
// Added bold html tag replacement for type ahead
private static string EscapeSearchTerm(string input)
{
input = Regex.Replace(input, @"<b>", "");
input = Regex.Replace(input, @"</b>", "");
input = Regex.Replace(input, @"\+", " ");
input = Regex.Replace(input, @"\-", " ");
input = Regex.Replace(input, @"\&", " ");
input = Regex.Replace(input, @"\|", " ");
input = Regex.Replace(input, @"\!", " ");
input = Regex.Replace(input, @"\(", " ");
input = Regex.Replace(input, @"\)", " ");
input = Regex.Replace(input, @"\{", " ");
input = Regex.Replace(input, @"\}", " ");
input = Regex.Replace(input, @"\[", " ");
input = Regex.Replace(input, @"\]", " ");
input = Regex.Replace(input, @"\^", " ");
input = Regex.Replace(input, @"\"""", " ");
input = Regex.Replace(input, @"\~", " ");
input = Regex.Replace(input, @"\*", " ");
input = Regex.Replace(input, @"?", " ");
input = Regex.Replace(input, @"\:", " ");
input = Regex.Replace(input, @"\\", " ");
return input;
}
public static string GenerateHighlightedText(Analyzer a, Query q, string docPart, string fieldName)
{
QueryScorer scorer = new QueryScorer(q, fieldName);
SimpleHTMLFormatter formatter = new SimpleHTMLFormatter("<mark><strong>", "</strong></mark>");
Highlighter highlighter = new Highlighter(formatter, scorer);
highlighter.TextFragmenter = (new SimpleFragmenter(int.MaxValue));
TokenStream stream = a.GetTokenStream(fieldName, docPart);
return highlighter.GetBestFragments(stream, docPart, 10, "...");
}
}
}
</pre>
<p><strong>Highlighting in the UI</strong></p>
<p>The changes to Index.razor to support highlighted text are very minimal. Since the Waffle Head and Waffle Body are being displayed in the search results already, the highlighting comes through in the HTML tags added by the GenerateHighlightedText() method. The only thing that needs to be done is to cast the text to markup using(MarkupString) so that the HTML tags aren't rendered literally. You can find this in lines 131 and 137 of the Index.razor code below.</p>
<pre data-enlighter-language="csharp">
@page "/"
@inject NavigationManager NavManager
<PageTitle>Prose Search</PageTitle>
<BSRow>
<BSCol Column="12" Padding="Padding.Small">
</BSCol>
</BSRow>
<EditForm Model="@searchModel" OnValidSubmit="@HandleSearch">
<DataAnnotationsValidator />
<BSRow Padding="Padding.None">
<BSCol Column="4" PaddingStart="Padding.Small">
<BlazoredTypeahead SearchMethod="HandleTypeAhead"
@bind-Value="searchModel.SearchText"
Placeholder="Enter Prose Text...">
<SelectedTemplate Context="searchText">
@((MarkupString)@searchText)
</SelectedTemplate>
<ResultTemplate Context="searchText">
@((MarkupString)@searchText)
</ResultTemplate>
</BlazoredTypeahead>
</BSCol>
<BSCol Column="1" Padding="Padding.None">
<BSButton type="Submit" Color="BSColor.Primary" PaddingTopAndBottom="Padding.Small">Search</BSButton>
</BSCol>
</BSRow>
</EditForm>
@if(@SearchText!=String.Empty)
{
<BSRow>
<BSCol Column="12" PaddingTop="Padding.Medium">
<div class="mb-12">
<div>Showing <b>@((Page*5)-4) - @(Math.Min(Page*5, SearchResultsCount))</b> out of <b>@SearchResultsCount</b> for: <i>@SearchText</i> </div>
</div>
</BSCol>
</BSRow>
<BSRow>
<BSCol Column="12" Padding="Padding.Small">
</BSCol>
</BSRow>
}
<BSRow>
<BSCol Column="3" PaddingLeftAndRight="Padding.Medium">
@if(@SearchResultsCount>0)
{
<BSRow class="h-100 border rounded">
<BSCol Column="12" PaddingLeftAndRight="Padding.Medium">
<div class="mb-12">
<BSRow Padding="Padding.ExtraSmall"></BSRow>
<BSRow class="text-muted"><b>Scholars</b></BSRow>
<BSRow Padding="Padding.ExtraSmall"></BSRow>
@if (@ScholarFacet.Count == 0)
{
@foreach (var _scholarFacet in @searchModel.FacetResults[0].LabelValues)
{
<BSRow Padding="Padding.ExtraSmall">
<BSCol Column="10" Align="Align.Start">
<BSLink href="javascript:void(0)" class="link-dark" style="text-decoration: none" OnClick="() => ScholarFilter(_scholarFacet)">@_scholarFacet.Label</BSLink>
</BSCol>
<BSCol Column="2" Align="Align.End">@_scholarFacet.Value</BSCol>
</BSRow>
}
}
else
{
<BSRow Padding="Padding.ExtraSmall">
<BSCol Column="12" Align="Align.Start">
<b>@ScholarFacet[0]</b>
(
<BSLink href="javascript:void(0)" class="link-primary" style="text-decoration: none" OnClick="ScholarRemove">Remove</BSLink>
)
</BSCol>
</BSRow>
}
<BSRow Padding="Padding.Small"></BSRow>
<BSRow class="text-muted"><b>Universities</b></BSRow>
<BSRow Padding="Padding.ExtraSmall"></BSRow>
@if (@UniversityFacet.Count == 0)
{
@foreach (var _universityFacet in @searchModel.FacetResults[1].LabelValues)
{
<BSRow Padding="Padding.ExtraSmall">
<BSCol Column="10" Align="Align.Start">
<BSLink href="javascript:void(0)" class="link-dark" style="text-decoration: none" OnClick="() => UniversityFilter(_universityFacet)">@_universityFacet.Label</BSLink>
</BSCol>
<BSCol Column="2" Align="Align.End">@_universityFacet.Value</BSCol>
</BSRow>
}
}
else
{
<BSRow Padding="Padding.ExtraSmall">
<BSCol Column="12" Align="Align.Start">
<b>@UniversityFacet[0]</b>
(
<BSLink href="javascript:void(0)" class="link-primary" style="text-decoration: none" OnClick="UniversityRemove">Remove</BSLink>
)
</BSCol>
</BSRow>
}
</div>
</BSCol>
</BSRow>
}
</BSCol>
<BSCol Column="9">
@if(@SearchResultsCount>0)
{
<BSRow>
<BSCol Column="12">
<div class="mb-12">
<BSListGroup>
@foreach (var result in @searchModel.CurrentPageSearchResults)
{
<BSListGroupItem>
<div class="d-flex w-100 justify-content-between">
<h5 class="mb-1">@((MarkupString)@result.WaffleHead)</h5>
</div>
<div>
<BSBadge Color="BSColor.Primary">@result.WaffleScholar</BSBadge>
<BSBadge Color="BSColor.Secondary">@result.WaffleUniversity</BSBadge>
</div>
<p class="mb-1">@((MarkupString)@result.WaffleBody)</p>
</BSListGroupItem>
}
</BSListGroup>
</div>
</BSCol>
</BSRow>
}
</BSCol>
</BSRow>
<BSRow>
<BSCol Column="12">
<BSRow>
<BSCol Column="12" Padding="Padding.Small">
</BSCol>
</BSRow>
<BSRow>
<BSCol Column="3">
</BSCol>
<BSCol Column="9">
<BSRow @onclick="UpdatePage">
<BSCol Column="12">
<div class="mb-12">
@if(@PageCount>1)
{
<BSPagination Pages=@PageCount @bind-Value="Page"/>
}
</div>
</BSCol>
</BSRow>
</BSCol>
</BSRow>
</BSCol>
</BSRow>
<BSRow>
<BSCol Column="12" Padding="Padding.Large">
</BSCol>
</BSRow>
@code {
private SearchModel searchModel = new SearchModel();
[Parameter]
public int Page {get; set;} = 1;
[Parameter]
public int PageCount {get; set;} = 0;
[Parameter]
public string SearchText {get; set;} = string.Empty;
[Parameter]
public int SearchResultsCount {get; set;} = 0;
[Parameter]
public List<string> ScholarFacet {get; set;} = new List<string>();
[Parameter]
public List<string> UniversityFacet {get; set;} = new List<string>();
private void HandleSearch()
{
ScholarFacet.Clear();
UniversityFacet.Clear();
Page = 1;
UpdatePage();
}
private async Task<IEnumerable<String>> HandleTypeAhead(string searchText)
{
List<String> SResult = SearchEngine.SearchAhead(searchText);
return await Task.FromResult(SResult.Where(x => x.ToLower().Contains(searchText.ToLower())).ToList());
}
private void ScholarFilter(Lucene.Net.Facet.LabelAndValue _scholarFacet)
{
ScholarFacet.Clear();
ScholarFacet.Add(_scholarFacet.Label);
Page = 1;
UpdatePage();
}
private void ScholarRemove()
{
ScholarFacet.Clear();
Page = 1;
UpdatePage();
}
private void UniversityFilter(Lucene.Net.Facet.LabelAndValue _universityFacet)
{
UniversityFacet.Clear();
UniversityFacet.Add(_universityFacet.Label);
Page = 1;
UpdatePage();
}
private void UniversityRemove()
{
UniversityFacet.Clear();
Page = 1;
UpdatePage();
}
private void UpdatePage()
{
if (searchModel.SearchText is not null && searchModel.SearchText.Length > 0)
{
searchModel = SearchEngine.FacetedSearch(searchModel.SearchText, Page, ScholarFacet, UniversityFacet);
SearchResultsCount = searchModel.ResultsCount;
PageCount = searchModel.PageCount;
SearchText = searchModel.SearchText;
}
else
{
NavManager.NavigateTo("/");
}
}
}
</pre>
<p><strong>Highlighting - Enablement</strong></p>
<p>Finally, the Lucene.Net.Hihglighter library is added to the project in the .csproj file.</p>
<pre data-enlighter-language="xml">
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Blazored.Typeahead" Version="4.7.0" />
<PackageReference Include="BlazorStrap" Version="5.0.106" />
<PackageReference Include="Bogus" Version="34.0.2" />
<PackageReference Include="Lucene.Net" Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.Analysis.Common" Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.Facet" Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.Highlighter" Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.QueryParser" Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.Suggest" Version="4.8.0-beta00016" />
<PackageReference Include="Microsoft.Extensions.Configuration" Version="6.0.1" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="6.0.0" />
<PackageReference Include="WaffleGenerator.Bogus" Version="4.2.1" />
</ItemGroup>
</Project>
</pre>
<p>In this final installment of my Blazor + Lucene.Net series, we'll be adding highlights for the search terms found in the header and body text of each of our results. The implementation of highlighting makes use of the Lucene.Net.Highlighter library, plugging this library into a simple method that can be used as a filter for search results to highlight key terms.</p>
/2022/11/lucene-blazor-part-4-faceting
Lucene + Blazor, Part 4: Faceting
2022-11-20T00:00:00Z
<p>In this fourth installment of my Blazor + Lucene.Net series, we'll make the most significant updates to the classes, search function and UI to date. Facets enrich the query responses, enabling users to further tune the search results along specific, pre-defined vectors. The implementation of facets uses the Lucene.Net.Facet library to implement an additional faceted index over two attributes -- Scholars and Universities -- which are applied on top of the WaffleText class and data.</p>
<p>The code and code narrative below reflects the changes that have been made on top of the first 3 posts. All <a href="https://github.com/thbst16/dotnet-lucene-search/tree/main/4-Faceting">source code is available online</a> for this faceting post.</p>
<p><strong>Sample App</strong></p>
<p>The sample application let's you search over 3,000 waffle text entries, returning paginated search results. Auto-complete functionality provides suggestion for the most relevant search terms in the waffle text index. On top of the search results, two attributes (Scholars and Universities) are available as facets. These facets can be drilled into or removed, shaping the results of the query appropriately. The site is available online at <a href="https://dotnet-lucene-search.azurewebsites.net/">https://dotnet-lucene-search.azurewebsites.net/</a></p>
<p><img src="https://s3.amazonaws.com/s3.beckshome.com/20221120-dotnet-lucene-faceting.jpeg" alt="Faceting" /></p>
<p><strong>Faceted Search</strong></p>
<p>There are three significant changes to the SearchEngine.cs class:</p>
<ol>
<li>Modification of the GetData() data generation function to generate Scholar and University facets for each of the WaffleText entries. The classes to support these facets are covered in subsequent sections.</li>
<li>Changes to the Index() method to add a new index for the facets and index the new facet fields.</li>
<li>Replacement of the default Search() method with a more advanced FacetedSearch() method that supports drill-down queries into each of the facets.</li>
</ol>
<p>The heart of the search ahead function is included in a new method (SearchAhead) in the SearchEngine.cs file. The function creates a dictionary of terms on top of the search index and then searches that for the input text, returning an ordered set of results of words starting with the typed letters. To get this to work, I had to add a new field to the index (HeadBody) because there doesn't seem to be a way to apply a MultiFieldQueryParser over the LuceneDictionary of terms.</p>
<pre data-enlighter-language="csharp">
using Bogus;
using Lucene.Net.Analysis;
using Lucene.Net.Analysis.Standard;
using Lucene.Net.QueryParsers.Classic;
using Lucene.Net.Documents;
using Lucene.Net.Facet;
using Lucene.Net.Facet.Taxonomy;
using Lucene.Net.Facet.Taxonomy.Directory;
using Lucene.Net.Index;
using Lucene.Net.Search;
using Lucene.Net.Search.Spell;
using Lucene.Net.Search.Suggest.Analyzing;
using Lucene.Net.Store;
using Lucene.Net.Util;
using System.Text.RegularExpressions;
namespace search.Shared
{
public class SearchEngine{
public static List<WaffleText> Data {get; set;}
private static RAMDirectory _indexDirectory;
private static RAMDirectory _facetDirectory;
public static IndexWriter indexWriter { get; set; }
public static DirectoryTaxonomyWriter taxoWriter { get; set; }
private static FacetsConfig facetConfig = new FacetsConfig();
public static void GetData(int Rand, int WaffleCount)
{
Randomizer.Seed = new Random(Rand);
var testWaffles = new Faker<WaffleText>()
.RuleFor(wt => wt.GUID, f => Guid.NewGuid().ToString())
.RuleFor(
property: wt => wt.WaffleHead,
setter: (f, wt) => f.WaffleTitle())
.RuleFor(
property: wt => wt.WaffleBody,
setter: (f, wt) => f.WaffleText(
paragraphs: 2,
includeHeading: false))
.RuleFor(wt => wt.WaffleScholar, f => f.PickRandom<WaffleScholar>())
.RuleFor(wt => wt.WaffleUniversity, f => f.PickRandom<WaffleUniversity>());
var waffles = testWaffles.Generate(WaffleCount);
Data = new List<WaffleText>();
foreach(WaffleText wt in waffles)
{
Data.Add(wt);
}
}
public static void Index()
{
const LuceneVersion lv = LuceneVersion.LUCENE_48;
Analyzer a = new StandardAnalyzer(lv);
_indexDirectory = new RAMDirectory();
_facetDirectory = new RAMDirectory();
var config = new IndexWriterConfig(lv, a);
indexWriter = new IndexWriter(_indexDirectory, config);
taxoWriter = new DirectoryTaxonomyWriter(_facetDirectory);
var doc = new Document();
foreach (WaffleText wt in Data)
{
doc = new Document();
doc.Add(new StringField("GUID", wt.GUID, Field.Store.YES));
doc.Add(new TextField("WaffleHead", wt.WaffleHead, Field.Store.YES));
doc.Add(new TextField("WaffleBody", wt.WaffleBody, Field.Store.YES));
doc.Add(new TextField("HeadBody", wt.WaffleHead + " " + wt.WaffleBody, Field.Store.YES));
doc.Add(new TextField("WaffleScholarTxt", wt.WaffleScholar.ToString(), Field.Store.YES));
doc.Add(new TextField("WaffleUniversityTxt", wt.WaffleUniversity.ToString(), Field.Store.YES));
doc.Add(new FacetField("WaffleScholar", wt.WaffleScholar.ToString()));
doc.Add(new FacetField("WaffleUniversity", wt.WaffleUniversity.ToString()));
indexWriter.AddDocument(facetConfig.Build(taxoWriter, doc));
}
indexWriter.Commit();
taxoWriter.Commit();
}
public static void Dispose()
{
indexWriter.Dispose();
taxoWriter.Dispose();
_indexDirectory.Dispose();
_facetDirectory.Dispose();
}
public static SearchModel FacetedSearch(string input, int page, List<string> scholarDrillDowns = null, List<string> universityDrillDowns = null)
{
const LuceneVersion lv = LuceneVersion.LUCENE_48;
Analyzer a = new StandardAnalyzer(lv);
string[] fnames = { "GUID", "WaffleHead", "WaffleBody" };
var multiFieldQP = new MultiFieldQueryParser(lv, fnames, a);
string _input = EscapeSearchTerm(input.Trim());
Query query = multiFieldQP.Parse(_input);
// Add drill down query
DrillDownQuery ddq = new DrillDownQuery(facetConfig, query);
if (scholarDrillDowns is not null)
{
foreach (string scholar in scholarDrillDowns)
{
ddq.Add("WaffleScholar", scholar);
}
}
if (universityDrillDowns is not null)
{
foreach (string university in universityDrillDowns)
{
ddq.Add("WaffleUniversity", university);
}
}
using (DirectoryReader indexReader = DirectoryReader.Open(_indexDirectory))
using (TaxonomyReader taxoReader = new DirectoryTaxonomyReader(_facetDirectory))
{
IndexSearcher searcher = new IndexSearcher(indexReader);
// Execute document search and return collection of WaffleText class
ScoreDoc[] docs = searcher.Search(ddq, null, 1000).ScoreDocs;
var waffles = new List<WaffleText>();
int first = (page-1)*5;
int last = first + 5;
for (int i = first; i < last && i < docs.Length; i++)
{
Document doc = searcher.Doc(docs[i].Doc);
WaffleText _waffle = new WaffleText();
_waffle.GUID = doc.Get("GUID");
_waffle.WaffleHead = doc.Get("WaffleHead");
_waffle.WaffleBody = doc.Get("WaffleBody");
_waffle.WaffleScholar = (WaffleScholar)Enum.Parse(typeof(WaffleScholar), doc.Get("WaffleScholarTxt"));
_waffle.WaffleUniversity = (WaffleUniversity)Enum.Parse(typeof(WaffleUniversity), doc.Get("WaffleUniversityTxt"));
waffles.Add(_waffle);
}
var returnModel = new SearchModel();
returnModel.CurrentPageSearchResults = waffles;
returnModel.SearchText = _input;
returnModel.ResultsCount = docs.Length;
returnModel.PageCount = (int)Math.Ceiling(docs.Length/5.0);
returnModel.CurrentPage = page;
// Execute facets search and return collection of FacetResults class
FacetsCollector fc = new FacetsCollector();
FacetsCollector.Search(searcher, ddq, 100, fc);
IList<FacetResult> results = new List<FacetResult>();
Facets facets = new FastTaxonomyFacetCounts(taxoReader, facetConfig, fc);
results.Add(facets.GetTopChildren(100, "WaffleScholar"));
results.Add(facets.GetTopChildren(100, "WaffleUniversity"));
returnModel.FacetResults = results;
return returnModel;
}
}
public static List<string> SearchAhead(string input)
{
const LuceneVersion lv = LuceneVersion.LUCENE_48;
Analyzer a = new StandardAnalyzer(lv);
var dirReader = DirectoryReader.Open(_indexDirectory);
LuceneDictionary dictionary = new LuceneDictionary(dirReader, "HeadBody");
RAMDirectory _d = new RAMDirectory();
AnalyzingInfixSuggester analyzingSuggester = new AnalyzingInfixSuggester(lv, _d, a);
analyzingSuggester.Build(dictionary);
var lookupResultList = analyzingSuggester.DoLookup(input.Trim(), false, 9);
List<string> returnModel = new List<string>();
foreach(var result in lookupResultList)
{
returnModel.Add(result.Key);
}
return returnModel;
dirReader.Dispose();
}
// Lucene supports escaping the following chars: + - && || ! ( ) { } [ ] ^ " ~ * ? : \
// To make it easier, I remove / replace the text altogether
// Added bold html tag replacement for type ahead
private static string EscapeSearchTerm(string input)
{
input = Regex.Replace(input, @"<b>", "");
input = Regex.Replace(input, @"</b>", "");
input = Regex.Replace(input, @"\+", " ");
input = Regex.Replace(input, @"\-", " ");
input = Regex.Replace(input, @"\&", " ");
input = Regex.Replace(input, @"\|", " ");
input = Regex.Replace(input, @"\!", " ");
input = Regex.Replace(input, @"\(", " ");
input = Regex.Replace(input, @"\)", " ");
input = Regex.Replace(input, @"\{", " ");
input = Regex.Replace(input, @"\}", " ");
input = Regex.Replace(input, @"\[", " ");
input = Regex.Replace(input, @"\]", " ");
input = Regex.Replace(input, @"\^", " ");
input = Regex.Replace(input, @"\"""", " ");
input = Regex.Replace(input, @"\~", " ");
input = Regex.Replace(input, @"\*", " ");
input = Regex.Replace(input, @"?", " ");
input = Regex.Replace(input, @"\:", " ");
input = Regex.Replace(input, @"\\", " ");
return input;
}
}
}
</pre>
<p><strong>WaffleText Support for Facets</strong></p>
<p>The base WaffleText class in WaffleText.cs changes to add support for the WaffleScholar and WaffleUniversity facets. Both of these are implemented as separate Enums.</p>
<pre data-enlighter-language="csharp">
namespace search.Shared
{
public class WaffleText
{
public string? GUID { get; set; }
public string? WaffleHead { get; set; }
public string? WaffleBody { get; set; }
public WaffleScholar? WaffleScholar { get; set; }
public WaffleUniversity? WaffleUniversity { get; set; }
public WaffleText() {}
public WaffleText(string _guid, string _waffleHead, string _waffleBody, WaffleScholar _waffleScholar, WaffleUniversity _waffleUniversity)
{
GUID = _guid;
WaffleHead = _waffleHead;
WaffleBody = _waffleBody;
WaffleScholar = _waffleScholar;
WaffleUniversity = _waffleUniversity;
}
}
}
</pre>
<p><strong>Addition of Enums to Support Faceting</strong></p>
<p>The WaffleScholar and WaffleUniversity enums were both added to support the new facets.</p>
<pre data-enlighter-language="csharp">
namespace search.Shared
{
public enum WaffleScholar
{
Freud,
Copernicus,
Erasmus,
Descartes,
Einstein,
Newton,
Goethe,
Confucius
}
}
</pre>
<pre data-enlighter-language="csharp">
namespace search.Shared
{
public enum WaffleUniversity
{
MIT,
Cambridge,
Stanford,
Oxford,
Harvard,
Caltech,
UCL,
Penn,
Edinburgh,
Princeton
}
}
</pre>
<p><strong>Faceted User Interface</strong></p>
<p>The changes to Index.razor to support a faceted UI have been the most significant changes to the UI to date. Try it out and see how the search results, pagination and search result summary are all dynamically updated based upon the selection / un-selection of individual facets.</p>
<p>The following changes were necessary:</p>
<ul>
<li>Changes to the search result wording, showing the number of results returned and displayed out of the total results.</li>
<li>Addition of a new display area housing the facets. This display area iterates over the facets returned from a search and displays them. It also makes calls to the new ScholarFilter() / ScholarRemove() and UniversityFilter() / UniversityRemove() methods to apply the facets and update the search results.</li>
<li>Addition of facet badges for each of the search results to visually show the facets associated with each of the individual results of the search.</li>
</ul>
<pre data-enlighter-language="csharp">
@page "/"
@inject NavigationManager NavManager
<PageTitle>Prose Search</PageTitle>
<BSRow>
<BSCol Column="12" Padding="Padding.Small">
</BSCol>
</BSRow>
<EditForm Model="@searchModel" OnValidSubmit="@HandleSearch">
<DataAnnotationsValidator />
<BSRow Padding="Padding.None">
<BSCol Column="4" PaddingStart="Padding.Small">
<BlazoredTypeahead SearchMethod="HandleTypeAhead"
@bind-Value="searchModel.SearchText"
Placeholder="Enter Prose Text...">
<SelectedTemplate Context="searchText">
@((MarkupString)@searchText)
</SelectedTemplate>
<ResultTemplate Context="searchText">
@((MarkupString)@searchText)
</ResultTemplate>
</BlazoredTypeahead>
</BSCol>
<BSCol Column="1" Padding="Padding.None">
<BSButton type="Submit" Color="BSColor.Primary" PaddingTopAndBottom="Padding.Small">Search</BSButton>
</BSCol>
</BSRow>
</EditForm>
@if(@SearchText!=String.Empty)
{
<BSRow>
<BSCol Column="12" PaddingTop="Padding.Medium">
<div class="mb-12">
<div>Showing <b>@((Page*5)-4) - @(Math.Min(Page*5, SearchResultsCount))</b> out of <b>@SearchResultsCount</b> for: <i>@SearchText</i> </div>
</div>
</BSCol>
</BSRow>
<BSRow>
<BSCol Column="12" Padding="Padding.Small">
</BSCol>
</BSRow>
}
<BSRow>
<BSCol Column="3" PaddingLeftAndRight="Padding.Medium">
@if(@SearchResultsCount>0)
{
<BSRow class="h-100 border rounded">
<BSCol Column="12" PaddingLeftAndRight="Padding.Medium">
<div class="mb-12">
<BSRow Padding="Padding.ExtraSmall"></BSRow>
<BSRow class="text-muted"><b>Scholars</b></BSRow>
<BSRow Padding="Padding.ExtraSmall"></BSRow>
@if (@ScholarFacet.Count == 0)
{
@foreach (var _scholarFacet in @searchModel.FacetResults[0].LabelValues)
{
<BSRow Padding="Padding.ExtraSmall">
<BSCol Column="10" Align="Align.Start">
<BSLink href="javascript:void(0)" class="link-dark" style="text-decoration: none" OnClick="() => ScholarFilter(_scholarFacet)">@_scholarFacet.Label</BSLink>
</BSCol>
<BSCol Column="2" Align="Align.End">@_scholarFacet.Value</BSCol>
</BSRow>
}
}
else
{
<BSRow Padding="Padding.ExtraSmall">
<BSCol Column="12" Align="Align.Start">
<b>@ScholarFacet[0]</b>
(
<BSLink href="javascript:void(0)" class="link-primary" style="text-decoration: none" OnClick="ScholarRemove">Remove</BSLink>
)
</BSCol>
</BSRow>
}
<BSRow Padding="Padding.Small"></BSRow>
<BSRow class="text-muted"><b>Universities</b></BSRow>
<BSRow Padding="Padding.ExtraSmall"></BSRow>
@if (@UniversityFacet.Count == 0)
{
@foreach (var _universityFacet in @searchModel.FacetResults[1].LabelValues)
{
<BSRow Padding="Padding.ExtraSmall">
<BSCol Column="10" Align="Align.Start">
<BSLink href="javascript:void(0)" class="link-dark" style="text-decoration: none" OnClick="() => UniversityFilter(_universityFacet)">@_universityFacet.Label</BSLink>
</BSCol>
<BSCol Column="2" Align="Align.End">@_universityFacet.Value</BSCol>
</BSRow>
}
}
else
{
<BSRow Padding="Padding.ExtraSmall">
<BSCol Column="12" Align="Align.Start">
<b>@UniversityFacet[0]</b>
(
<BSLink href="javascript:void(0)" class="link-primary" style="text-decoration: none" OnClick="UniversityRemove">Remove</BSLink>
)
</BSCol>
</BSRow>
}
</div>
</BSCol>
</BSRow>
}
</BSCol>
<BSCol Column="9">
@if(@SearchResultsCount>0)
{
<BSRow>
<BSCol Column="12">
<div class="mb-12">
<BSListGroup>
@foreach (var result in @searchModel.CurrentPageSearchResults)
{
<BSListGroupItem>
<div class="d-flex w-100 justify-content-between">
<h5 class="mb-1">@result.WaffleHead</h5>
</div>
<div>
<BSBadge Color="BSColor.Primary">@result.WaffleScholar</BSBadge>
<BSBadge Color="BSColor.Secondary">@result.WaffleUniversity</BSBadge>
</div>
<p class="mb-1">@result.WaffleBody</p>
</BSListGroupItem>
}
</BSListGroup>
</div>
</BSCol>
</BSRow>
}
</BSCol>
</BSRow>
<BSRow>
<BSCol Column="12">
<BSRow>
<BSCol Column="12" Padding="Padding.Small">
</BSCol>
</BSRow>
<BSRow>
<BSCol Column="3">
</BSCol>
<BSCol Column="9">
<BSRow @onclick="UpdatePage">
<BSCol Column="12">
<div class="mb-12">
@if(@PageCount>1)
{
<BSPagination Pages=@PageCount @bind-Value="Page"/>
}
</div>
</BSCol>
</BSRow>
</BSCol>
</BSRow>
</BSCol>
</BSRow>
<BSRow>
<BSCol Column="12" Padding="Padding.Large">
</BSCol>
</BSRow>
@code {
private SearchModel searchModel = new SearchModel();
[Parameter]
public int Page {get; set;} = 1;
[Parameter]
public int PageCount {get; set;} = 0;
[Parameter]
public string SearchText {get; set;} = string.Empty;
[Parameter]
public int SearchResultsCount {get; set;} = 0;
[Parameter]
public List<string> ScholarFacet {get; set;} = new List<string>();
[Parameter]
public List<string> UniversityFacet {get; set;} = new List<string>();
private void HandleSearch()
{
ScholarFacet.Clear();
UniversityFacet.Clear();
Page = 1;
UpdatePage();
}
private async Task<IEnumerable<String>> HandleTypeAhead(string searchText)
{
List<String> SResult = SearchEngine.SearchAhead(searchText);
return await Task.FromResult(SResult.Where(x => x.ToLower().Contains(searchText.ToLower())).ToList());
}
private void ScholarFilter(Lucene.Net.Facet.LabelAndValue _scholarFacet)
{
ScholarFacet.Clear();
ScholarFacet.Add(_scholarFacet.Label);
Page = 1;
UpdatePage();
}
private void ScholarRemove()
{
ScholarFacet.Clear();
Page = 1;
UpdatePage();
}
private void UniversityFilter(Lucene.Net.Facet.LabelAndValue _universityFacet)
{
UniversityFacet.Clear();
UniversityFacet.Add(_universityFacet.Label);
Page = 1;
UpdatePage();
}
private void UniversityRemove()
{
UniversityFacet.Clear();
Page = 1;
UpdatePage();
}
private void UpdatePage()
{
if (searchModel.SearchText is not null && searchModel.SearchText.Length > 0)
{
searchModel = SearchEngine.FacetedSearch(searchModel.SearchText, Page, ScholarFacet, UniversityFacet);
SearchResultsCount = searchModel.ResultsCount;
PageCount = searchModel.PageCount;
SearchText = searchModel.SearchText;
}
else
{
NavManager.NavigateTo("/");
}
}
}
</pre>
<p><strong>Facet - Enablement</strong></p>
<p>Finally, the Lucene.Net.Facet library is added to the project in the .csproj file.</p>
<pre data-enlighter-language="xml">
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Blazored.Typeahead" Version="4.7.0" />
<PackageReference Include="BlazorStrap" Version="5.0.106" />
<PackageReference Include="Bogus" Version="34.0.2" />
<PackageReference Include="Lucene.Net" Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.Analysis.Common" Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.Facet" Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.QueryParser" Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.Suggest" Version="4.8.0-beta00016" />
<PackageReference Include="Microsoft.Extensions.Configuration" Version="6.0.1" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="6.0.0" />
<PackageReference Include="WaffleGenerator.Bogus" Version="4.2.1" />
</ItemGroup>
</Project>
</pre>
<p>In this fourth installment of my Blazor + Lucene.Net series, we'll make the most significant updates to the classes, search function and UI to date. Facets enrich the query responses, enabling users to further tune the search results along specific, pre-defined vectors. The implementation of facets uses the Lucene.Net.Facet library to implement an additional faceted index over two attributes -- Scholars and Universities -- which are applied on top of the WaffleText class and data.</p>
/2022/11/lucene-blazor-part-3-auto-complete
Lucene + Blazor, Part 3: Auto Complete
2022-11-12T00:00:00Z
<p>In this third installment of my Blazor + Lucene.Net series, I'll start tackling some advanced Lucene functionality, namely auto-complete. For advanced Lucene work, the most important lessons is <b>don't roll your own</b> functionality. If you go to the <a href="https://lucenenet.apache.org/docs/4.8.0-beta00007/api/Lucene.Net/overview.html">docs for the Lucene.Net API</a>, you'll see that a ton of additional functionality is built into Lucene via modules. Modules exist for faceting, highlighting, spatial search and autosuggest, amongst others. Lots of the examples and StackOverflow answers are roll-your-own solutions -- don't do it!!</p>
<p>Specific to this installment around Auto Complete, I employed two specific libraries:</p>
<ol>
<li><a href="https://lucenenet.apache.org/docs/4.8.0-beta00007/api/Lucene.Net.Suggest/overview.html">Lucene.Net Suggest</a> - The auto complete / auto suggest library includes the methods to index the data for autosuggest and then a number of suggester algorithms to query the index.</li>
<li><a href="https://github.com/Blazored/Typeahead">Blazored.Typeahead</a> - A drop-in Blazor control for type-ahead that accommodates things like debouncing time before executing searches.</li>
</ol>
<p>The code and code narrative below reflects the changes that have been made on top of posts 1 and 2. All <a href="https://github.com/thbst16/dotnet-lucene-search/tree/main/3-AutoComplete">source code is available online</a> for this auto-complete post.</p>
<p><strong>Sample App</strong></p>
<p>The sample application let's you search over 3,000 waffle text entries, returning paginated search results. Auto-complete functionality provides suggestion for the most relevant search terms in the waffle text index. The site is available online at <a href="https://dotnet-lucene-search.azurewebsites.net/">https://dotnet-lucene-search.azurewebsites.net/</a></p>
<p><img src="https://s3.amazonaws.com/s3.beckshome.com/20221111-dotnet-lucene-auto-complete.jpeg" alt="Auto Complete"></p>
<p><strong>Auto Complete Function</strong></p>
<p>The heart of the search ahead function is included in a new method (SearchAhead) in the SearchEngine.cs file. The function creates a dictionary of terms on top of the search index and then searches that for the input text, returning an ordered set of results of words starting with the typed letters. To get this to work, I had to add a new field to the index (HeadBody) because there doesn't seem to be a way to apply a MultiFieldQueryParser over the LuceneDictionary of terms.</p>
<pre data-enlighter-language="csharp">using Bogus;
using Lucene.Net.Analysis;
using Lucene.Net.Analysis.Standard;
using Lucene.Net.QueryParsers.Classic;
using Lucene.Net.Documents;
using Lucene.Net.Index;
using Lucene.Net.Search;
using Lucene.Net.Search.Spell;
using Lucene.Net.Search.Suggest.Analyzing;
using Lucene.Net.Store;
using Lucene.Net.Util;
using System.Text.RegularExpressions;
namespace search.Shared
{
public class SearchEngine{
public static List<waffletext> Data {get; set;}
private static RAMDirectory _directory;
public static IndexWriter Writer { get; set; }
public static void GetData(int Rand, int WaffleCount)
{
Randomizer.Seed = new Random(Rand);
var testWaffles = new Faker<waffletext>()
.RuleFor(wt => wt.GUID, f => Guid.NewGuid().ToString())
.RuleFor(
property: wt => wt.WaffleHead,
setter: (f, wt) => f.WaffleTitle())
.RuleFor(
property: wt => wt.WaffleBody,
setter: (f, wt) => f.WaffleText(
paragraphs: 2,
includeHeading: false));
var waffles = testWaffles.Generate(WaffleCount);
Data = new List<waffletext>();
foreach(WaffleText wt in waffles)
{
Data.Add(wt);
}
}
public static void Index()
{
const LuceneVersion lv = LuceneVersion.LUCENE_48;
Analyzer a = new StandardAnalyzer(lv);
_directory = new RAMDirectory();
var config = new IndexWriterConfig(lv, a);
Writer = new IndexWriter(_directory, config);
var guidField = new StringField("GUID", "", Field.Store.YES);
var headField = new TextField("WaffleHead", "", Field.Store.YES);
var bodyField = new TextField("WaffleBody", "", Field.Store.YES);
// Hoping for better way to do this than indexing combined field
var headBody = new TextField("HeadBody", "", Field.Store.YES);
var d = new Document()
{
guidField,
headField,
bodyField,
headBody
};
foreach (WaffleText wt in Data)
{
guidField.SetStringValue(wt.GUID);
headField.SetStringValue(wt.WaffleHead);
bodyField.SetStringValue(wt.WaffleBody);
headBody.SetStringValue(wt.WaffleHead + " " + wt.WaffleBody);
Writer.AddDocument(d);
}
Writer.Commit();
}
public static void Dispose()
{
Writer.Dispose();
_directory.Dispose();
}
public static SearchModel Search(string input, int page)
{
const LuceneVersion lv = LuceneVersion.LUCENE_48;
Analyzer a = new StandardAnalyzer(lv);
var dirReader = DirectoryReader.Open(_directory);
var searcher = new IndexSearcher(dirReader);
string[] waffles = { "GUID", "WaffleHead", "WaffleBody" };
var multiFieldQP = new MultiFieldQueryParser(lv, waffles, a);
string _input = EscapeSearchTerm(input.Trim());
Query query = multiFieldQP.Parse(_input);
ScoreDoc[] docs = searcher.Search(query, null, 1000).ScoreDocs;
var returnModel = new SearchModel();
returnModel.CurrentPageSearchResults = new List<waffletext>();
returnModel.SearchText = _input;
returnModel.ResultsCount = docs.Length;
returnModel.PageCount = (int)Math.Ceiling(docs.Length/5.0);
returnModel.CurrentPage = page;
int first = (page-1)*5;
int last = first + 5;
for (int i = first; i < last && i < docs.Length; i++)
{
Document d = searcher.Doc(docs[i].Doc);
WaffleText _localWaffle = new WaffleText();
_localWaffle.GUID = d.Get("GUID");
_localWaffle.WaffleHead = d.Get("WaffleHead");
_localWaffle.WaffleBody = d.Get("WaffleBody");
returnModel.CurrentPageSearchResults.Add(_localWaffle);
}
dirReader.Dispose();
return returnModel;
}
public static List<string> SearchAhead(string input)
{
const LuceneVersion lv = LuceneVersion.LUCENE_48;
Analyzer a = new StandardAnalyzer(lv);
var dirReader = DirectoryReader.Open(_directory);
LuceneDictionary dictionary = new LuceneDictionary(dirReader, "HeadBody");
RAMDirectory _d = new RAMDirectory();
AnalyzingInfixSuggester analyzingSuggester = new AnalyzingInfixSuggester(lv, _d, a);
analyzingSuggester.Build(dictionary);
var lookupResultList = analyzingSuggester.DoLookup(input.Trim(), false, 9);
List<string> returnModel = new List<string>();
foreach(var result in lookupResultList)
{
returnModel.Add(result.Key);
}
return returnModel;
dirReader.Dispose();
}
// Lucene supports escapting the following chars: + - && || ! ( ) { } [ ] ^ " ~ * ? : \
// To make it easier, I remove / replace the text altogether
// Added bold html tag replacement for type ahead
private static string EscapeSearchTerm(string input)
{
input = Regex.Replace(input, @"<b>", "");
input = Regex.Replace(input, @"</b>", "");
input = Regex.Replace(input, @"\+", " ");
input = Regex.Replace(input, @"\-", " ");
input = Regex.Replace(input, @"\&", " ");
input = Regex.Replace(input, @"\|", " ");
input = Regex.Replace(input, @"\!", " ");
input = Regex.Replace(input, @"\(", " ");
input = Regex.Replace(input, @"\)", " ");
input = Regex.Replace(input, @"\{", " ");
input = Regex.Replace(input, @"\}", " ");
input = Regex.Replace(input, @"\[", " ");
input = Regex.Replace(input, @"\]", " ");
input = Regex.Replace(input, @"\^", " ");
input = Regex.Replace(input, @"\""", " ");
input = Regex.Replace(input, @"\~", " ");
input = Regex.Replace(input, @"\*", " ");
input = Regex.Replace(input, @"?", " ");
input = Regex.Replace(input, @"\:", " ");
input = Regex.Replace(input, @"\\", " ");
return input;
}
}
}
</string></string></string></waffletext></waffletext></waffletext></waffletext></pre>
<p><strong>Auto Complete User Interface</strong></p>
<p>The auto complete user interface is the other place where meaningful changes were required to accommodate auto-complete functionality. These changes include the addition of the Blazored.TypeAhead control to the page and the new HandleTypeAhead method that invokes the search function.</p>
<pre data-enlighter-language="csharp">@page "/"
<pagetitle>Prose Search</pagetitle>
<bsrow>
<bscol column="12" padding="Padding.Small">
</bscol>
</bsrow>
<editform model="@searchModel" onvalidsubmit="@HandleSearch">
<dataannotationsvalidator>
<bsrow padding="Padding.None">
<bscol column="4" paddingstart="Padding.Small">
<blazoredtypeahead searchmethod="HandleTypeAhead" @bind-value="searchModel.SearchText" placeholder="Enter Prose Text...">
<selectedtemplate context="searchText">
@((MarkupString)@searchText)
</selectedtemplate>
<resulttemplate context="searchText">
@((MarkupString)@searchText)
</resulttemplate>
</blazoredtypeahead>
</bscol>
<bscol column="1" padding="Padding.None">
<bsbutton type="Submit" color="BSColor.Primary" paddingtopandbottom="Padding.Small">Search</bsbutton>
</bscol>
</bsrow>
</dataannotationsvalidator></editform>
@if(@SearchText!=String.Empty)
{
<bsrow>
<bscol column="12" paddingtop="Padding.Medium">
<div class="mb-12">
@if(@SearchResultsCount==1)
{
<div><b>@SearchResultsCount Result</b></div>
}
else
{
<div><b>@SearchResultsCount Results</b></div>
}
</div>
</bscol>
</bsrow>
}
@if(@SearchResultsCount>0)
{
<bsrow>
<bscol column="12" padding="Padding.Small">
</bscol>
</bsrow>
<bsrow>
<bscol column="9">
<div class="mb-9">
<bslistgroup>
@foreach (var result in @searchModel.CurrentPageSearchResults)
{
<bslistgroupitem>
<div class="d-flex w-100 justify-content-between">
<h5 class="mb-1">@result.WaffleHead</h5>
</div>
<p class="mb-1">@result.WaffleBody</p>
</bslistgroupitem>
}
</bslistgroup>
</div>
</bscol>
</bsrow>
@if(@PageCount>1)
{
<bsrow>
<bscol column="12" padding="Padding.Small">
</bscol>
</bsrow>
<bsrow @onclick="UpdatePage">
<bscol column="9">
<div class="mb-9">
<bspagination pages="@PageCount" @bind-value="Page">
</bspagination></div>
</bscol>
</bsrow>
}
<bsrow>
<bscol column="12" padding="Padding.Large">
</bscol>
</bsrow>
}
@code {
private SearchModel searchModel = new SearchModel();
[Parameter]
public int Page {get; set;} = 1;
[Parameter]
public int PageCount {get; set;} = 0;
[Parameter]
public string SearchText {get; set;} = string.Empty;
[Parameter]
public int SearchResultsCount {get; set;} = 0;
private void HandleSearch()
{
searchModel = SearchEngine.Search(searchModel.SearchText, 1);
SearchResultsCount = searchModel.ResultsCount;
PageCount = searchModel.PageCount;
SearchText = searchModel.SearchText;
Page = 1;
}
private async Task<ienumerable<string>> HandleTypeAhead(string searchText)
{
List<string> SResult = SearchEngine.SearchAhead(searchText);
IEnumerable<string> AResult = new List<string>();
if (!SResult.Contains(searchText))
{
AResult = SResult.Prepend("<b>"+searchText+"</b>");
}
else
{
AResult = SResult;
}
return await Task.FromResult(AResult.Where(x => x.ToLower().Contains(searchText.ToLower())).ToList());
}
private void UpdatePage()
{
searchModel = SearchEngine.Search(searchModel.SearchText, Page);
}
}
</string></string></string></ienumerable<string></pre>
<p><strong>Type Ahead Control - Front End</strong></p>
<p>The Blazored.Typeahead control needs to be added to the _Layout.cshtml file, brining along the requisite JS and CSS files.</p>
<pre data-enlighter-language="html">@using Microsoft.AspNetCore.Components.Web
@namespace search.Pages
@addTagHelper *, Microsoft.AspNetCore.Mvc.TagHelpers
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<base href="/~/">
<link rel="stylesheet" href="/css/bootstrap/bootstrap.min.css">
<link href="/css/site.css" rel="stylesheet">
<link href="/search.styles.css" rel="stylesheet">
<link href="/search.styles.css" rel="stylesheet">
<link href="/_content/Blazored.Typeahead/blazored-typeahead.css" rel="stylesheet">
<component type="typeof(HeadOutlet)" render-mode="ServerPrerendered">
@RenderBody()
<div id="blazor-error-ui">
<environment include="Staging,Production">
An error has occurred. This application may no longer respond until reloaded.
</environment>
<environment include="Development">
An unhandled exception has occurred. See browser dev tools for details.
</environment>
<a href="" class="reload">Reload</a>
<a class="dismiss">🗙</a>
</div>
<script src="/_content/BlazorStrap/popper.min.js"></script>
<pre><code><script src="_content/BlazorStrap/blazorstrap.js"></script>
<script src="_content/Blazored.Typeahead/blazored-typeahead.js"></script>
<script src="_framework/blazor.server.js"></script>
</code></pre>
</component></pre>
<p>And the BlazorStrap library using statement needs to be added to the _Imports.razor file.</p>
<pre data-enlighter-language="csharp">@using System.Net.Http
@using Blazored.Typeahead
@using BlazorStrap
@using Microsoft.AspNetCore.Authorization
@using Microsoft.AspNetCore.Components.Authorization
@using Microsoft.AspNetCore.Components.Forms
@using Microsoft.AspNetCore.Components.Routing
@using Microsoft.AspNetCore.Components.Web
@using Microsoft.AspNetCore.Components.Web.Virtualization
@using Microsoft.JSInterop
@using search
@using search.Shared
</pre>
<p><strong>Type Ahead - Enablement</strong></p>
<p>Finally, the Blazored.Typeahead and Lucene.Net.Suggest libraries are added to the project in the .csproj file.</p>
<pre data-enlighter-language="xml"><project sdk="Microsoft.NET.Sdk.Web">
<propertygroup>
<targetframework>net6.0</targetframework>
<nullable>enable</nullable>
<implicitusings>enable</implicitusings>
</propertygroup>
<itemgroup>
<packagereference include="Blazored.Typeahead" version="4.7.0">
<packagereference include="BlazorStrap" version="5.0.106">
<packagereference include="Bogus" version="34.0.2">
<packagereference include="Lucene.Net" version="4.8.0-beta00016">
<packagereference include="Lucene.Net.Analysis.Common" version="4.8.0-beta00016">
<packagereference include="Lucene.Net.QueryParser" version="4.8.0-beta00016">
<packagereference include="Lucene.Net.Suggest" version="4.8.0-beta00016">
<packagereference include="Microsoft.Extensions.Configuration" version="6.0.1">
<packagereference include="Microsoft.Extensions.Configuration.Json" version="6.0.0">
<packagereference include="WaffleGenerator.Bogus" version="4.2.1">
</packagereference></packagereference></packagereference></packagereference></packagereference></packagereference></packagereference></packagereference></packagereference></packagereference></itemgroup>
</project>
</pre>
<p>In this third installment of my Blazor + Lucene.Net series, I'll start tackling some advanced Lucene functionality, namely auto-complete. For advanced Lucene work, the most important lessons is <b>don't roll your own</b> functionality. If you go to the <a href="https://lucenenet.apache.org/docs/4.8.0-beta00007/api/Lucene.Net/overview.html">docs for the Lucene.Net API</a>, you'll see that a ton of additional functionality is built into Lucene via modules. Modules exist for faceting, highlighting, spatial search and autosuggest, amongst others. Lots of the examples and StackOverflow answers are roll-your-own solutions -- don't do it!!</p>
/2022/11/lucene-blazor-part-2-results-paging
Lucene + Blazor, Part 2: Results Paging
2022-11-05T00:00:00Z
<p>In the <a href="https://beckshome.com/2022/10/lucene-blazor-part-1-basic-search">first installment of this series</a>, we looked at returning results from a limited pool of items in a Lucene full text index. In this second installment, we significantly increase the number of generated items (3,000, by default) and add a numbered paging system, as used by the main commercial search engines and search sites.</p>
<p>The code and code narrative below reflects the changes that have been made since the first post. All <a href="https://github.com/thbst16/dotnet-lucene-search/tree/main/2-ResultsPaging">source code is available online</a> for this results paging post.</p>
<p><strong>Sample App</strong></p>
<p>The sample application generates 3,000 waffle text records with the exact count being configurable and stored in the appsettings.json file. These waffle items can be searched and return in paginated form with a default page size of 5 records (not configurable). Additional character escaping / nulling has been added to remove characters from searches prior to passing them to the search engine. The site is available online at <a href="https://dotnet-lucene-search.azurewebsites.net/">https://dotnet-lucene-search.azurewebsites.net/</a></p>
<p><img src="https://s3.amazonaws.com/s3.beckshome.com/20221104-dotnet-lucene-search-pagination.jpeg" alt="Results Paging" /></p>
<p><strong>Dynamic Configuration - Settings</strong></p>
<p>The dynamic configuration settings, specifically the size of the waffle text corpus to be generated and the random seed initializer used for generation, are stored in the appsettings.json file and read at runtime.</p>
<pre data-enlighter-language="json">
{
"Logging": {
"LogLevel": {
"Default": "Information",
"Microsoft.AspNetCore": "Warning"
}
},
"AllowedHosts": "*",
"BogusConfig": {
"Rand": "11784",
"WaffleCount": "3000"
}
}
</pre>
<p><strong>Dynamic Configuration - Enablement</strong></p>
<p>The Microsoft.Extensions libraries are added to the project in the .csproj file to enable dynamic, JSON-based configuration.</p>
<pre data-enlighter-language="xml">
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="BlazorStrap" Version="5.0.106" />
<PackageReference Include="Bogus" Version="34.0.2" />
<PackageReference Include="Lucene.Net" Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.Analysis.Common" Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.QueryParser" Version="4.8.0-beta00016" />
<PackageReference Include="Microsoft.Extensions.Configuration" Version="6.0.1" />
<PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="6.0.0" />
<PackageReference Include="WaffleGenerator.Bogus" Version="4.2.1" />
</ItemGroup>
</Project>
</pre>
<p><strong>Dynamic Configuration - Activation</strong></p>
<p>The program's dynamic configuration is implemented in the Program.cs file. The configuration file is open, settings are read and then passed dynamically to the GetData() method of the engine, which generates the sample data.</p>
<pre data-enlighter-language="csharp">
using Microsoft.AspNetCore.Components;
using Microsoft.AspNetCore.Components.Web;
using Microsoft.Extensions.Configuration;
using search.Shared;
using BlazorStrap;
var builder = WebApplication.CreateBuilder(args);
// Add services to the container.
builder.Services.AddRazorPages();
builder.Services.AddServerSideBlazor();
builder.Services.AddBlazorStrap();
// Setup configuration
var cfgBuilder = new ConfigurationBuilder().AddJsonFile($"appsettings.json", true, true);
var config = cfgBuilder.Build();
// Search engine setup
SearchEngine.GetData(Int32.Parse(config["BogusConfig:Rand"]), Int32.Parse(config["BogusConfig:WaffleCount"]));
SearchEngine.Index();
var app = builder.Build();
// Configure the HTTP request pipeline.
if (!app.Environment.IsDevelopment())
{
app.UseExceptionHandler("/Error");
// The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseRouting();
app.MapBlazorHub();
app.MapFallbackToPage("/_Host");
app.Run();
</pre>
<p><strong>Pagination - Model Enablement</strong></p>
<p>To enable pagination, additional attributes are added to the SearchModel class to allow for the total count of pages and current page for a specific search.</p>
<pre data-enlighter-language="csharp">
using System.ComponentModel.DataAnnotations;
namespace search.Shared
{
public class SearchModel{
[Required]
public string SearchText {get; set;}
public int ResultsCount {get; set;}
public int PageCount {get; set;}
public int CurrentPage {get; set;}
public List<WaffleText> CurrentPageSearchResults {get; set;}
}
}
</pre>
<p><strong>Pagination - Implementation</strong></p>
<p>Pagination is implemented on the back end in the SearchEngine.cs class. The Search method signature and method have been changed significantly from the original post to enable paginated searches. Also, an EscapeSearchTerm function has been added to remove specific characters from the search text. This function is applied to search input within the Search method.</p>
<pre data-enlighter-language="csharp">
using Bogus;
using Lucene.Net.Analysis;
using Lucene.Net.Analysis.Standard;
using Lucene.Net.QueryParsers.Classic;
using Lucene.Net.Documents;
using Lucene.Net.Index;
using Lucene.Net.Search;
using Lucene.Net.Store;
using Lucene.Net.Util;
using System.Text.RegularExpressions;
namespace search.Shared
{
public class SearchEngine{
public static List<WaffleText> Data {get; set;}
private static RAMDirectory _directory;
public static IndexWriter Writer { get; set; }
public static void GetData(int Rand, int WaffleCount)
{
Randomizer.Seed = new Random(Rand);
var testWaffles = new Faker<WaffleText>()
.RuleFor(wt => wt.GUID, f => Guid.NewGuid().ToString())
.RuleFor(
property: wt => wt.WaffleHead,
setter: (f, wt) => f.WaffleTitle())
.RuleFor(
property: wt => wt.WaffleBody,
setter: (f, wt) => f.WaffleText(
paragraphs: 2,
includeHeading: false));
var waffles = testWaffles.Generate(WaffleCount);
Data = new List<WaffleText>();
foreach(WaffleText wt in waffles)
{
Data.Add(wt);
}
}
public static void Index()
{
const LuceneVersion lv = LuceneVersion.LUCENE_48;
Analyzer a = new StandardAnalyzer(lv);
_directory = new RAMDirectory();
var config = new IndexWriterConfig(lv, a);
Writer = new IndexWriter(_directory, config);
var guidField = new StringField("GUID", "", Field.Store.YES);
var headField = new TextField("WaffleHead", "", Field.Store.YES);
var bodyField = new TextField("WaffleBody", "", Field.Store.YES);
var d = new Document()
{
guidField,
headField,
bodyField
};
foreach (WaffleText wt in Data)
{
guidField.SetStringValue(wt.GUID);
headField.SetStringValue(wt.WaffleHead);
bodyField.SetStringValue(wt.WaffleBody);
Writer.AddDocument(d);
}
Writer.Commit();
}
public static void Dispose()
{
Writer.Dispose();
_directory.Dispose();
}
public static SearchModel Search(string input, int page)
{
const LuceneVersion lv = LuceneVersion.LUCENE_48;
Analyzer a = new StandardAnalyzer(lv);
var dirReader = DirectoryReader.Open(_directory);
var searcher = new IndexSearcher(dirReader);
string[] waffles = { "GUID", "WaffleHead", "WaffleBody" };
var multiFieldQP = new MultiFieldQueryParser(lv, waffles, a);
string _input = EscapeSearchTerm(input.Trim());
Query query = multiFieldQP.Parse(_input);
ScoreDoc[] docs = searcher.Search(query, null, 1000).ScoreDocs;
var returnModel = new SearchModel();
returnModel.CurrentPageSearchResults = new List<WaffleText>();
returnModel.SearchText = _input;
returnModel.ResultsCount = docs.Length;
returnModel.PageCount = (int)Math.Ceiling(docs.Length/5.0);
returnModel.CurrentPage = page;
int first = (page-1)*5;
int last = first + 5;
for (int i = first; i < last && i < docs.Length; i++)
{
Document d = searcher.Doc(docs[i].Doc);
WaffleText _localWaffle = new WaffleText();
_localWaffle.GUID = d.Get("GUID");
_localWaffle.WaffleHead = d.Get("WaffleHead");
_localWaffle.WaffleBody = d.Get("WaffleBody");
returnModel.CurrentPageSearchResults.Add(_localWaffle);
}
dirReader.Dispose();
return returnModel;
}
// Lucene supports escaping the following chars: + - && || ! ( ) { } [ ] ^ " ~ * ? : \
// To make it easier, I remove / replace
private static string EscapeSearchTerm(string input)
{
input = Regex.Replace(input, @"\+", " ");
input = Regex.Replace(input, @"\-", " ");
input = Regex.Replace(input, @"\&", " ");
input = Regex.Replace(input, @"\|", " ");
input = Regex.Replace(input, @"\!", " ");
input = Regex.Replace(input, @"\(", " ");
input = Regex.Replace(input, @"\)", " ");
input = Regex.Replace(input, @"\{", " ");
input = Regex.Replace(input, @"\}", " ");
input = Regex.Replace(input, @"\[", " ");
input = Regex.Replace(input, @"\]", " ");
input = Regex.Replace(input, @"\^", " ");
input = Regex.Replace(input, @"\"", " ");
input = Regex.Replace(input, @"\~", " ");
input = Regex.Replace(input, @"\*", " ");
input = Regex.Replace(input, @"?", " ");
input = Regex.Replace(input, @"\:", " ");
input = Regex.Replace(input, @"\\", " ");
return input;
}
}
}
</pre>
<p><strong>Pagination - User Interface</strong></p>
<p>Finally, the front-end pagination is added to the Index.razor class. All of this is made much easier through the presence of a very capable <a href="https://blazorstrap.io/V5/components/pagination">BlazorStrap pagination component</a>.</p>
<pre data-enlighter-language="csharp">
@page "/"
<PageTitle>Prose Search</PageTitle>
<BSRow>
<BSCol Column="12" Padding="Padding.Small">
</BSCol>
</BSRow>
<EditForm Model="@searchModel" OnValidSubmit="@HandleSearch">
<DataAnnotationsValidator />
<BSRow>
<BSCol Column="6" Padding="Padding.Small">
<div class="input-group mb-3">
<InputText class="form-control" placeholder="Enter Prose Text" @bind-Value="searchModel.SearchText" />
<BSButton type="Submit" Color="BSColor.Primary">Search</BSButton>
</div>
</BSCol>
</BSRow>
</EditForm>
@if(@SearchText!=String.Empty)
{
<BSRow>
<BSCol Column="12">
<div class="mb-12">
@if(@SearchResultsCount==1)
{
<div>@SearchResultsCount Result</div>
}
else
{
<div>@SearchResultsCount Results</div>
}
</div>
</BSCol>
</BSRow>
}
@if(@SearchResultsCount>0)
{
<BSRow>
<BSCol Column="12" Padding="Padding.Small">
</BSCol>
</BSRow>
<BSRow>
<BSCol Column="9">
<div class="mb-9">
<BSListGroup>
@foreach (var result in @searchModel.CurrentPageSearchResults)
{
<BSListGroupItem>
<div class="d-flex w-100 justify-content-between">
<h5 class="mb-1">@result.WaffleHead</h5>
</div>
<p class="mb-1">@result.WaffleBody</p>
</BSListGroupItem>
}
</BSListGroup>
</div>
</BSCol>
</BSRow>
@if(@PageCount>1)
{
<BSRow>
<BSCol Column="12" Padding="Padding.Small">
</BSCol>
</BSRow>
<BSRow @onclick="UpdatePage">
<BSCol Column="9">
<div class="mb-9">
<BSPagination Pages=@PageCount @bind-Value="Page"/>
</div>
</BSCol>
</BSRow>
}
<BSRow>
<BSCol Column="12" Padding="Padding.Large">
</BSCol>
</BSRow>
}
@code {
private SearchModel searchModel = new SearchModel();
[Parameter]
public int Page {get; set;} = 1;
[Parameter]
public int PageCount {get; set;} = 0;
[Parameter]
public string SearchText {get; set;} = string.Empty;
[Parameter]
public int SearchResultsCount {get; set;} = 0;
private void HandleSearch()
{
searchModel = SearchEngine.Search(searchModel.SearchText, 1);
SearchResultsCount = searchModel.ResultsCount;
PageCount = searchModel.PageCount;
SearchText = searchModel.SearchText;
Page = 1;
}
private void UpdatePage()
{
searchModel = SearchEngine.Search(searchModel.SearchText, Page);
}
}
</pre>
<p>In the <a href="https://beckshome.com/2022/10/lucene-blazor-part-1-basic-search">first installment of this series</a>, we looked at returning results from a limited pool of items in a Lucene full text index. In this second installment, we significantly increase the number of generated items (3,000, by default) and add a numbered paging system, as used by the main commercial search engines and search sites.</p>
/2022/10/lucene-blazor-part-1-basic-search
Lucene + Blazor, Part 1: Basic Search
2022-10-30T00:00:00Z
<p>Lucene.NET is a C# port of the Java Lucene search engine library. Lucene.NET provides robust search and index capabilities enhanced by a wide array of support packages (e.g. auto-suggest, faceting) that enable the creation of robust search facilities within .NET applications.</p>
<p>While <a href="https://code-maze.com/how-to-implement-lucene-dotnet/">useful and recent tutorials</a> exist on using Lucene.NET in a command line context, there is a dearth of tutorials on using Lucene.NET in a web context, especially with the current Blazor framework. This tutorial fills that gap, with this existing article being the first in a series that will illustrate the buildout of a Blazor-based search site.</p>
<p><strong>Sample App</strong></p>
<p>The sample application generates sample waffle text, indexes this text and provides a web search front end. The image below illustrates the basic user interface. This site is available online at <a href="https://dotnet-lucene-search.azurewebsites.net/">https://dotnet-lucene-search.azurewebsites.net/</a> with the source code for this tutorial at <a href="https://github.com/thbst16/dotnet-lucene-search/tree/main/1-BasicSearch">https://github.com/thbst16/dotnet-lucene-search/tree/main/1-BasicSearch</a>.</p>
<p><img src="https://s3.amazonaws.com/s3.beckshome.com/20221029-dotnet-lucene-search-basic.jpeg" alt="BlazorCrud Home Page" /></p>
<p><strong>Required Libraries</strong></p>
<p>Several NuGet packages are required to run the application, as illustrated in the search.csproj file below. These packages include the latest pre-releases of the Lucene.Net framework and supporting packages for the search engine, BlazorStrap for Blazor-based UI components and the Bogus data generation library.</p>
<pre data-enlighter-language="xml">
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net6.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="BlazorStrap" Version="5.0.106" />
<PackageReference Include="Bogus" Version="34.0.2" />
<PackageReference Include="Lucene.Net" Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.Analysis.Common" Version="4.8.0-beta00016" />
<PackageReference Include="Lucene.Net.QueryParser" Version="4.8.0-beta00016" />
<PackageReference Include="WaffleGenerator.Bogus" Version="4.2.1" />
</ItemGroup>
</Project>
</pre>
<p><strong>Waffle Text</strong></p>
<p>The search engine app will be generating and searching on <a href="https://www.red-gate.com/simple-talk/development/dotnet-development/the-waffle-generator/">waffle text</a>. Support for waffle text generation is provided by an ancillary Bogus library. The definition of the waffle text class is provided in WaffleText.cs.</p>
<pre data-enlighter-language="csharp">
namespace search.Shared
{
public class WaffleText
{
public string? GUID { get; set; }
public string? WaffleHead { get; set; }
public string? WaffleBody { get; set; }
public WaffleText() {}
public WaffleText(string _guid, string _waffleHead, string _waffleBody)
{
GUID = _guid;
WaffleHead = _waffleHead;
WaffleBody = _waffleBody;
}
}
}
</pre>
<p><strong>Search Model</strong></p>
<p>The model for handling the search inputs, results count and collection of waffles is defined in the SearchModel.cs class.</p>
<pre data-enlighter-language="csharp">
using System.ComponentModel.DataAnnotations;
namespace search.Shared
{
public class SearchModel{
[Required]
public string SearchText {get; set;}
public int ResultsCount {get; set;}
public List<WaffleText> SearchResults {get; set;}
}
}
</pre>
<p><strong>Search Engine</strong></p>
<p>The magic of the app is handled in the SearchEngine.cs class. This class interacts with the Lucene.NET search engine library and the Bogus library to facilitate the generation, indexing and search of waffle data. The SearchEngine.cs class has 3 major methods:</p>
<ol>
<li><b>GetData.</b> Uses the Bogus data generation library to generate 50 pseudo-random waffle text items.</li>
<li><b>Index.</b> Uses the Lucene.NET search engine library to index the generated waffle text items for search.</li>
<li><b>Search.</b> Provides the search function that searches over the indexed waffle text using a scored search.</li>
</ol>
<p>Both the GetData and Index methods are called during program startup (from the Program.cs file). The Search method is invoked from the Blazor UI with the search text passed in from the user's input.</p>
<pre data-enlighter-language="csharp">
using Bogus;
using Lucene.Net.Analysis;
using Lucene.Net.Analysis.Standard;
using Lucene.Net.QueryParsers.Classic;
using Lucene.Net.Documents;
using Lucene.Net.Index;
using Lucene.Net.Search;
using Lucene.Net.Store;
using Lucene.Net.Util;
namespace search.Shared
{
public class SearchEngine{
public static List<WaffleText> Data {get; set;}
private static RAMDirectory _directory;
public static IndexWriter Writer { get; set; }
public static void GetData()
{
Randomizer.Seed = new Random(11784);
var testWaffles = new Faker<WaffleText>()
.RuleFor(wt => wt.GUID, f => Guid.NewGuid().ToString())
.RuleFor(
property: wt => wt.WaffleHead,
setter: (f, wt) => f.WaffleTitle())
.RuleFor(
property: wt => wt.WaffleBody,
setter: (f, wt) => f.WaffleText(
paragraphs: 2,
includeHeading: false));
var waffles = testWaffles.Generate(50);
Data = new List<WaffleText>();
foreach(WaffleText wt in waffles)
{
Data.Add(wt);
}
}
public static void Index()
{
const LuceneVersion lv = LuceneVersion.LUCENE_48;
Analyzer a = new StandardAnalyzer(lv);
_directory = new RAMDirectory();
var config = new IndexWriterConfig(lv, a);
Writer = new IndexWriter(_directory, config);
var guidField = new StringField("GUID", "", Field.Store.YES);
var headField = new TextField("WaffleHead", "", Field.Store.YES);
var bodyField = new TextField("WaffleBody", "", Field.Store.YES);
var d = new Document()
{
guidField,
headField,
bodyField
};
foreach (WaffleText wt in Data)
{
guidField.SetStringValue(wt.GUID);
headField.SetStringValue(wt.WaffleHead);
bodyField.SetStringValue(wt.WaffleBody);
Writer.AddDocument(d);
}
Writer.Commit();
}
public static void Dispose()
{
Writer.Dispose();
_directory.Dispose();
}
public static List<WaffleText> Search(string input)
{
const LuceneVersion lv = LuceneVersion.LUCENE_48;
Analyzer a = new StandardAnalyzer(lv);
var dirReader = DirectoryReader.Open(_directory);
var searcher = new IndexSearcher(dirReader);
string[] waffles = { "GUID", "WaffleHead", "WaffleBody" };
var multiFieldQP = new MultiFieldQueryParser(lv, waffles, a);
Query query = multiFieldQP.Parse(input.Trim());
ScoreDoc[] docs = searcher.Search(query, null, 1000).ScoreDocs;
var results = new List<WaffleText>();
for (int i = 0; i < docs.Length; i++)
{
Document d = searcher.Doc(docs[i].Doc);
WaffleText _localWaffle = new WaffleText();
_localWaffle.GUID = d.Get("GUID");
_localWaffle.WaffleHead = d.Get("WaffleHead");
_localWaffle.WaffleBody = d.Get("WaffleBody");
results.Add(_localWaffle);
}
dirReader.Dispose();
return results;
}
}
}
</pre>
<p><strong>Search Page</strong></p>
<p>The last piece of the puzzle is the user-facing Blazor search page that takes the user search input, invokes the engine and returns the search results. All of this functionality is contained in the Blazor page Index.razor.</p>
<pre data-enlighter-language="csharp">
@page "/"
<PageTitle>Prose Search</PageTitle>
<BSRow>
<BSCol Column="12" Padding="Padding.Small">
</BSCol>
</BSRow>
<EditForm Model="@searchModel" OnValidSubmit="@HandleSearch">
<DataAnnotationsValidator />
<BSRow>
<BSCol Column="6" Padding="Padding.Small">
<div class="input-group mb-3">
<InputText class="form-control" placeholder="Enter Prose Text" @bind-Value="searchModel.SearchText" />
<BSButton type="Submit" Color="BSColor.Primary">Search</BSButton>
</div>
</BSCol>
</BSRow>
</EditForm>
@if(@SearchText!=String.Empty)
{
<BSRow>
<BSCol Column="12">
<div class="mb-12">
@if(@SearchResultsCount==1)
{
<div>@SearchResultsCount Result</div>
}
else
{
<div>@SearchResultsCount Results</div>
}
</div>
</BSCol>
</BSRow>
}
@if(@SearchResultsCount>0)
{
<BSRow>
<BSCol Column="12" Padding="Padding.Small">
</BSCol>
</BSRow>
<BSRow>
<BSCol Column="9">
<div class="mb-9">
<BSListGroup>
@foreach (var result in @searchModel.SearchResults)
{
<BSListGroupItem>
<div class="d-flex w-100 justify-content-between">
<h5 class="mb-1">@result.WaffleHead</h5>
</div>
<p class="mb-1">@result.WaffleBody</p>
</BSListGroupItem>
}
</BSListGroup>
</div>
</BSCol>
</BSRow>
<BSRow>
<BSCol Column="12" Padding="Padding.Large">
</BSCol>
</BSRow>
}
@code {
private SearchModel searchModel = new SearchModel();
[Parameter]
public string SearchText {get; set;} = string.Empty;
[Parameter]
public int SearchResultsCount {get; set;} = 0;
private void HandleSearch()
{
searchModel.SearchResults = SearchEngine.Search(searchModel.SearchText);
searchModel.ResultsCount = searchModel.SearchResults.Count;
SearchResultsCount = searchModel.ResultsCount;
SearchText = searchModel.SearchText;
}
}
</pre>
<p>Lucene.NET is a C# port of the Java Lucene search engine library. Lucene.NET provides robust search and index capabilities enhanced by a wide array of support packages (e.g. auto-suggest, faceting) that enable the creation of robust search facilities within .NET applications.</p>
/2022/10/dotnet-lightweight-databases
Dotnet Lightweight Databases
2022-10-10T00:00:00Z
<p>My popular <a href="https://github.com/thbst16/dotnet-blazor-crud">dotnet-blazor-crud</a> project has long used the <a href="https://exceptionnotfound.net/ef-core-inmemory-asp-net-core-store-database/">EF Core InMemory database provider</a> for data persistence. While this hasn't caused me any issues -- ever, I've been aspiring to move to a relational database versus an in-memory provider. SQLite is the obvious contender here, without going to a full out-of-process database. Getting started with SQLite and EF Core is <a href="https://www.koderdojo.com/blog/getting-started-with-entity-framework-core-and-sqlite">pretty easy</a>. The real question for me was how much work it would take to swap out the in memory provider for SQLite.</p>
<p>Since SQLite is a file-based database and BlazorCrud is a Docker-based solution, I had to decide whether or not to <a href="https://docs.docker.com/get-started/05_persisting_data/">persist the DB</a> outside of the container. I use container volumes in many of my other projects but, since one of my goals of BlazorCrud is to have a pure Docker distribution (that is, no dependencies on mounts or Docker Compose), long term persistence was out.</p>
<p>With that decision out of the way, the steps to swap the EF Core InMemory provider for SQLite (or vice-versa) are pretty simple:</p>
<ol>
<li><p><b>Update DB Package References.</b> This is a simple swap of the Microsoft.EntityFrameworkCore.Sqlite provider for the Microsoft.EntityFrameworkCore.InMemory provider. Alternately, you could leave both there and switch between in memory and SQLite using configuration switches.
<img src="https://s3.amazonaws.com/s3.beckshome.com/20221010-db-package-reference.jpg" alt="Update DB Package References" /></p>
</li>
<li><p><b>Update AppDbContext Options.</b> This simply involves swapping in the UseSqlLite method (with a pointer to a physical file) for the UseInMemoryDatabase method.</p>
<p><img src="https://s3.amazonaws.com/s3.beckshome.com/20221010-db-context.jpg" alt="Update AppDBContext Options" /></p>
</li>
<li><p><b>Ensure Database Deleted / Created.</b> The EnsureCreated() method is necessary to ensure that the database for the context exists. The method ensures the database exists but provides no assurances around schema compatibility with the EF model. For testing and prototyping such as with BlazorCrud, I've added the EnsureDeleted() method as well to make sure a new database is created (and then populated with Bogus data) on every app start.</p>
<p><img src="https://s3.amazonaws.com/s3.beckshome.com/20221010-db-delete-create.jpg" alt="Ensure Database Created Deleted" /></p>
</li>
</ol>
<p>My popular <a href="https://github.com/thbst16/dotnet-blazor-crud">dotnet-blazor-crud</a> project has long used the <a href="https://exceptionnotfound.net/ef-core-inmemory-asp-net-core-store-database/">EF Core InMemory database provider</a> for data persistence. While this hasn't caused me any issues -- ever, I've been aspiring to move to a relational database versus an in-memory provider. SQLite is the obvious contender here, without going to a full out-of-process database. Getting started with SQLite and EF Core is <a href="https://www.koderdojo.com/blog/getting-started-with-entity-framework-core-and-sqlite">pretty easy</a>. The real question for me was how much work it would take to swap out the in memory provider for SQLite.</p>
/2022/09/mermaid-in-statiq
Mermaid Diagrams in Statiq
2022-09-27T00:00:00Z
<p>According to the definition on the <a href="https://mermaid-js.github.io/mermaid/#/README">Mermaid site</a>, Mermaid is a JavaScript based diagramming and charting tool that renders Markdown-inspired text definitions to create and modify diagrams dynamically. In short, when creating online Markdown-based documentation or publications, like with GitHub or with static site generators that use Markdown to generate content, you can move from static images to dynamically generated diagrams. Mermaid currently supports diagrams such as flowcharts, sequence diagrams, class diagrams, ER diagrams and Gantt charts with more diagrams being added.</p>
<p>Adding Mermaid diagrams to a Statiq blog, like the one I use, is a pretty straightforward task. Dpvreony has further simplified this task with some great documentation specific to <a href="https://www.dpvreony.com/articles/mermaid-with-statiq/">using Mermaid diagrams with Statiq</a>. I can confirm that his documented approach works well and he identifies a couple of items where the reader can enhance the approach, including processing diagrams in a loop and working all of this into a CI/CD process.</p>
<p>I've provided a couple of examples below. As with everything on this blog, the source can be found on <a href="https://github.com/thbst16/dotnet-statiq-beckshome-blog">my Github repo for this blog</a>.</p>
<p><strong>Flowchart</strong></p>
<p>Mermaid enables you to use all the major flowchart shapes and to put together some pretty complex mappings, including cross-flow dependencies. The example below is a simplified technology selection process.</p>
<pre data-enlighter-language="md">
flowchart LR
A[Start] --> B(Collect financial business case details)
B --> C{Financial benefits to moving to cloud}
C -->|Yes| D[Select cloud service provider]
C -->|No| E[Remain on-premise]
</pre>
<img src="/img/mermaid/flowchart.svg"/>
<br/><br/>
<p><strong>Sequence Diagram</strong></p>
<p>Mermaid also pulls off sequence diagrams pretty well, as illustrated by the simplified sequence diagram below.</p>
<pre data-enlighter-language="md">
sequenceDiagram
User-->Application: Update My Address
Application-->AddressValidationAPI: Validate
break if address validation API call fails
Application-->User: show failure
end
Application-->AddressService: Update User Address
</pre>
<img src="/img/mermaid/sequence.svg"/>
<br/><br/>
<p><strong>Gantt Chart</strong></p>
<p>Gantt charts for project management can be created via Mermaid as well. Although not suited for large, complex Gantt views, the simplified Mermaid charts are great for overall project status and an overview of activities and key dependencies.</p>
<pre data-enlighter-language="md">
gantt
dateFormat YYYY-MM-DD
title Cloud Migration Mermaid Gantt Charts
excludes weekends
section Data Activities
Extract and prepare data :done, des1, 2021-01-05,2021-01-07
Validate data integrity :active, des2, 2021-01-08, 3d
Obfuscate non-production data : des3, after des2, 5d
Backup data : des4, after des3, 2d
section Critical Path Items
Review migration runbook :crit, done, 2021-01-05,24h
Migrate data to cloud :crit, done, after des1, 2d
Build and validate app servers :crit, active, 3d
Test application in cloud :crit, 5d
Pre-production walkthorugh :crit, 2d
Receive go-live approval :crit, 1d
Cloud go-live :milestone, 2021-01-26, 0d
section Documentation
Complete compliance documentation :active, a1, after des1, 3d
Review Compliance Documentation :after a1 , 20h
Revise and Approve Compliance Docs :doc1, after a1 , 48h
</pre>
<img src="/img/mermaid/gantt.svg"/>
<p>According to the definition on the <a href="https://mermaid-js.github.io/mermaid/#/README">Mermaid site</a>, Mermaid is a JavaScript based diagramming and charting tool that renders Markdown-inspired text definitions to create and modify diagrams dynamically. In short, when creating online Markdown-based documentation or publications, like with GitHub or with static site generators that use Markdown to generate content, you can move from static images to dynamically generated diagrams. Mermaid currently supports diagrams such as flowcharts, sequence diagrams, class diagrams, ER diagrams and Gantt charts with more diagrams being added.</p>
/2022/09/early-tech-publications
Early Tech Publications
2022-09-19T00:00:00Z
<p>In porting my blog to Statiq, I was forced to revisit a whole lot of content, most of which is more than a decade old. As part of this review, I cleaned up old broken links, of which there were many. I would estimate that 60% of the web links I had from my original blog pages were to sites or content that no longer existed. This says a lot about how the web has evolved and how much content turns over as time progresses.</p>
<p>While I removed external links that no longer existed, there were some links to content from early publications I had authored for Dr.Dobb's Journal and the Java Developers Journal. I had hesitated posting the original articles since they were historically available online. It looks like somewhere between 2015 and 2022, these sites ceased to exist. I decided to publish the full original articles online since this is the only way they remain available.</p>
<p><img src="https://s3.amazonaws.com/s3.beckshome.com/20220919-early-tech-publications.jpg" alt="Early Tech Publications" /></p>
<ul>
<li><a href="https://s3.amazonaws.com/s3.beckshome.com/20220911-continuous-integration-and-dotnet-part-1.pdf">Continuous Integration and .NET - Part 1 (99 KB)</a></li>
<li><a href="https://s3.amazonaws.com/s3.beckshome.com/20220911-continuous-integration-and-dotnet-part-2.pdf">Continuous Integration and .NET - Part 2 (99 KB)</a></li>
<li><a href="https://s3.amazonaws.com/s3.beckshome.com/20220911-java-active-authentication.pdf">Java Active Authentication (833 KB)</a></li>
<li><a href="https://s3.amazonaws.com/s3.beckshome.com/20220911-open-source-software-collaboratives.pdf">Open Source Software Collaboratives (550 KB)</a></li>
</ul>
<p>In porting my blog to Statiq, I was forced to revisit a whole lot of content, most of which is more than a decade old. As part of this review, I cleaned up old broken links, of which there were many. I would estimate that 60% of the web links I had from my original blog pages were to sites or content that no longer existed. This says a lot about how the web has evolved and how much content turns over as time progresses.</p>
/2022/09/beckshome-on-statiq
Beckshome on Statiq
2022-09-10T00:00:00Z
<p>After 7 years of being dormant, the beckshome.com blog hummed back to life over the past month. It started with <a href="/2006/06/das-blog-installation">a full .NET hosted blog</a>, <a href="/2011/09/web-hosting-provider-cutover">moved to Wordpress</a> and then was exported from Wordpress to create static content in 2015, which is where things stood for the past 7 years. I decided to go with a static site generator to modernize the site, specifically with <a href="https://www.statiq.dev/web">Statiq</a>, which is a .NET-based static site generator.</p>
<p>Once you get beyond the Hello World examples of getting Statiq stood up, there's a lot of work to be done to get a blog ported over. Some of this work is brute force conversion of years of blog posts into Markdown format. Some of it lies in the details of Statiq configuration. Rather than reinventing the wheel on this latter topic, I point to a <a href="https://www.techwatching.dev/posts/migrating-blog">great blog post on migrating to Statiq</a> along with two GitHub repositories containing Open Sourced versions of live Statiq sites -- from <a href="https://github.com/techwatching/techwatching.dev">Alexandre Nédélec</a> and from <a href="https://github.com/daveaglick/daveaglick">David Glick</a>, the creator of Statiq. I have also made <a href="https://github.com/thbst16/dotnet-statiq-beckshome-blog">my blog's repository on GitHub</a> public.</p>
<p>There are several lessons I learned beyond what is in the aforementioned blogs. I have highlighted and documented these items below:</p>
<h3>Post Destination Paths</h3>
<p>The Clean Blog site theme documentation <a href="https://github.com/statiqdev/CleanBlog#post-destination-path">provides an example</a> of using the _directory.yml file in the posts folder to output posts with computed, date-specific paths. The provided example does the trick for date-specific paths but still handles .MD path extensions. My changes below take care of both the date-based paths and .MD extension handling.</p>
<pre data-enlighter-language="csharp">DestinationPath: => $"{Document.GetDateTime("Published").ToString("yyyy/MM")}" + "/" + $"{Document.Destination.FileName}".Replace(".md", "") + ".html"
</pre>
<h3> Rewrite Rules for Hosting on Azure App Services</h3>
<p>To get the Restful Url routing working and mask the HTML file extension required a custom rewrite rule in a web.config file. Surprisingly, I needed to put in a rule and a custom MIME extension for RSS as well. Very App Service / IIS specific but necessary if you're using these platforms.</p>
<pre data-enlighter-language="xml"><configuration>
<system.webserver>
<staticcontent>
<mimemap fileextension=".rss" mimetype="application/rss+xml">
</mimemap></staticcontent>
<rewrite>
<rules>
<rule name="rss" stopprocessing="true">
<match url="^feed.rss$">
<action type="None">
</action></match></rule>
<rule name="html">
<match url="(.*)">
<conditions>
<add input="{REQUEST_FILENAME}" matchtype="IsFile" negate="true">
<add input="{REQUEST_FILENAME}" matchtype="IsDirectory" negate="true">
</add></add></conditions>
<action type="Rewrite" url="{R:1}.html">
</action></match></rule>
</rules>
</rewrite>
</system.webserver>
</configuration>
</pre>
<h3>Google Analytics</h3>
<p>If you're looking to add <a href="https://analytics.google.com/">Google Analytics</a>, or other web tracking products, this is really simple in Statiq. Add the Google Analytics javascript to the _head.cshtml file, which should be blank, and go to town.</p>
<pre data-enlighter-language="js"><!-- Google tag (gtag.js) -->
<script async="" src="https://www.googletagmanager.com/gtag/js?id=G-YOUR-CODE-HERE"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-YOUR-CODE-HERE');
</script>
</pre>
<h3>Giscus Commenting</h3>
<p>Adding <a href="https://giscus.app/">giscus commenting</a> is as easy as dropping some Javascript into the _post-footer.cshtml file. You can generate the specific javascript on the giscus site or replace the fillers below.</p>
<pre data-enlighter-language="js"><script src="https://giscus.app/client.js" data-repo="YOUR-GITHUB-REPO" data-repo-id="YOUR-REPO-ID" data-category="Announcements" data-category-id="YOUR-CATEGORY-ID" data-mapping="pathname" data-strict="0" data-reactions-enabled="1" data-emit-metadata="0" data-input-position="bottom" data-theme="preferred_color_scheme" data-lang="en" crossorigin="anonymous" async="">
</script>
</pre>
<h3>Sidebar Social Links</h3>
<p>Adding the social links below the tags in the sidebar is a two-step process. First, add a reference to the social-links partial in the _sidebar.cshtml file.</p>
<pre data-enlighter-language="csharp">@Html.Partial("_social-links")
</pre>
<p>And then adding the following details into the _social-links.cshtml file with your specific links.</p>
<pre data-enlighter-language="csharp"><hr class="dark">
<div class="text-center">
<a href="https://twitter.com/YOU-ON-TWITTER"><i class="fab fa-twitter" aria-hidden="true"></i></a>
<a href="https://www.linkedin.com/in/YOU-ON-LINKEDIN/"><i class="fab fa-linkedin" aria-hidden="true"></i></a>
<a href="https://www.facebook.com/YOU-ON-FACEBOOK"><i class="fab fa-facebook" aria-hidden="true"></i></a>
</div>
</pre>
<p>After 7 years of being dormant, the beckshome.com blog hummed back to life over the past month. It started with <a href="/2006/06/das-blog-installation">a full .NET hosted blog</a>, <a href="/2011/09/web-hosting-provider-cutover">moved to Wordpress</a> and then was exported from Wordpress to create static content in 2015, which is where things stood for the past 7 years. I decided to go with a static site generator to modernize the site, specifically with <a href="https://www.statiq.dev/web">Statiq</a>, which is a .NET-based static site generator.</p>
/2015/01/technology-links-january-11-2015
Technology Links - January 11, 2015
2015-01-11T00:00:00Z
<p>Couple of exciting things to link to from last week:</p>
<ul>
<li>Docker – O’Reilly released an awesome <a href="https://www.oreilly.com/library/view/introduction-to-docker/9781491916179/">Introduction to Docker video</a>. Total runtime is less than 2 hours. Definitely a recommended watch if you’re looking to get familiar with the Docker container. Free to those who have an O’Reilly Safari subscription.</li>
<li>Azure VM Images – Microsoft announces a host of <a href="https://devblogs.microsoft.com/visualstudio/azure-virtual-machine-images-for-visual-studio/">Azure Virtual Machine Images for Visual Studio</a>. A great way to get exposure to the other Visual Studio SKUs or run the newer versions of VS without having to deal with the headaches of an install.</li>
<li>App Development – <a href="https://medium.com/@carlosribas/how-hourstracker-earns-five-figures-a-month-on-the-app-store-85a20bb972eb">Awesome article</a> on how independent mobile app development works and can be quite lucrative.</li>
<li>Feature Toggles – Good course from Pluralsight on <a href="https://www.pluralsight.com/courses/dotnet-featuretoggle-implementing">implementing feature toggles in .NET</a>. Something architects should be aware but the process and patterns are not widely documented.</li>
</ul>
<p>Couple of exciting things to link to from last week:</p>