• Updated:
  • Featured post

Embeddings y Vector Search

Un ordenador no puede entender texto ni relaciones semánticas o significados entre palabras. Solo puede entender números. Este problema lo resolvemos mediante el uso de embeddings.

Un embedding es la representación de texto (en forma de números) en un espacio vectorial. Esto permite a los modelos de IA comparar y operar sobre el significado de las palabras.

flowchart TD
    A["perro"] --> B{{Modelo de embedding}}
    B --> C["[-0.003, 0.043, ..., -0.01]"]
    
    N1["(texto que queremos convertir)"]:::note --> A
    N2["(vectores con contenido semántico)"]:::note --> C
    
    classDef note fill:none,stroke:none,color:#777;    

Los vectores de cada palabra o documento capturan el significado semántico del texto.

  • perro estará cerca de mascota
  • contrato estará lejos de playa

Vector vs SQL databases

El problema con las bases de datos típicas es que solo buscan matches exactos. Si yo busco por coche solo me sacará las entradas que contengan coche.

En cambio las BBDD vectoriales pueden interpretar la semántica de las palabras mediante vectores. Si busco por coche puede sacarme valores como sedán, SUV, Land Rover, etc.

Las BBDD vectoriales son muy buenas cuando necesitamos buscar items similares por proximidad uno respecto al otro.

Algunos ejemplos de uso son:

  • buscar películas parecidas (Netflix)
  • Recomendadores de items parecidos en tiendas online (Amazon)
  • buscar canciones parecidas (Spotify)

Read More

Linq examples

Retrieve a simple subset of properties for all items in list

Usually, anon data types are used in the select clause to return a specific subset of properties for each object in the collection.

we have the employee class

public class Employee
{
	public int ID { get; set; }
	public string Name { get; set; }
	public int Age { get; set; }
	public string Address { get; set; }
}
List<Employee> employees = // ...

// we convert an Employee into an anon object type with a subset of properties
var employeeDetails = from emp in employees
	select new { Id = emp.ID, Name = emp.Name };

How to map List of classes

public List<MappedUser> MapListOfUsers(List<User> users)
{
	// method 1
	List<MappedUser> mappedUsers = users.ConvertAll(user => MapSingleUser(user));
	
	// method 2
	List<MappedUser> mappedUsers2 = 
		(from user in users select MapSingleUser(user)).ToList();
}

method to encapsulate mapping itself

private MappedUser MapSingleUser(User user)
{
 var mapped = new MappedUser
 {
	 Id = user.Id,
	 Name = user.Name,
	 Email = user.Email
 };
 return mapped;
}

This provides easier and more legible than doing a foreach to iterate everything.

How to filter list per properties (Where)

var adminUserTask = users
	.Where(user => "admin".Equals(user.type.ToLower()))
	.Select(async user => { return await ProcessAdmin(user);});
List<UserResults> results = (await Task.WhenAll(adminUserTask)).ToList();

filter by properties in a nullable list and return true if there’s any row that match. If the list is null, it returns false.

return response.results?.rows
	.Where(row => row.id == requestId && (row.owner == requestOwner || row.responsible == requestResponsible))
	.Any() ?? false;

another example

var task = response.results.rows
	.AsParallel()
	.Where(row => "specifictype".Equals(row.type.ToLower()))
	.Select(async row => {
		if(row.type.Equals("specificType"))
		{
			return await Something(row, id, log);
		} else 
		{
			return row;
		}
	});

How to select records based on another list (ids)

This is how to select a list of items, selecting them by id, based on another list of items

List<string> idList = // ...
var profiles = _context.UserProfiles
		.Where(userProf => idList.Contains(userProf.Id));

How to order based on a dynamic parameter

// Direction is an Enum w. values ASC or DESC
private List<Person> SortPersons(Direction direction, List<Person> persons, Func<Person, string> sortBy)
{
	if(direction.DESC)
	{
		return persons.OrderByDescending(sortBy.Invoke).ToList();
	} else 
	{
		return persons.OrderBy(sortBy.Invoke).ToList();
	}
}

How to use it

var sortedPersons = SortPersons(direction, persons, person => person.Name);

Single param lambda

remember that for lambdas with a single param where it comes from the same query, you don’t need to explicitely set it

this transforms

public List<int> FilterList(List<int> listToFilter, List<int> filterintList)
{
	return listToFilter.Where(number => filteringList.Contains(number)).ToList();
}

Methods worth a mention

Remove duplicates Distinct()

var task = response.results.rows
	.AsParallel()
	.Where(row => "specificType".Equals(row.type.ToLower()))
	.Select(async row => { return await Something(row, id, log);});

Get the difference of two lists Except()

var list = listToFilter
	.Except(filteringList)
	.ToList();

First() vs Single()

// gets the first element that matches and stops there
string list = listToFilter
	.First(s => s.StartsWith("ohno"));

// gets the matching element, or throws an exception if there's more than 1
string list = listToFilter
	.Single(s => s.StartsWith("ohno"));

Validaciones C#

C# tiene los DataAnnotation

[Required] hace que sea un campo obligatorio

[Required(ErrorMessage = "Nombre es obligatorio")]
public string NombreCategoria { get; set; }

[Required(ErrorMessage = "Orden es obligatorio")]
[Range(1, int.MaxValue, ErrorMessage = "El orden debe de ser mayor a cero")]
public int Orden {get; set; }

Se controla mediante el siguiente codigo en el controller. El código de ModelState es código base de un Controller.

if(ModelState.IsValid) 
{
	// code if everything's valid
}

C# coding style

Interfaces must start by capital I

public interface IDataService 
{ 
	public Task SendData(DataModel model);
}

We use PascalCase for:

  • classes’ name
  • methods’ name
  • public variables

We use CamelCase for:

  • private or internal field names (they must include the prefix _)
  • methods’ paramters
public class DataService
{
	const int TAX = 7;
	
	public bool IsValid { get; private set; };
	private IWorkerQueue _workerQueue;
	
	public async Task SendData(DataModel model)
	{
		string someValue = "";
		// ... whatever
	}
}

async methods must end by Async

public async Task<string> GetUrlAsync()
{
	// ... whatever
}

Reference(s)

https://learn.microsoft.com/es-es/dotnet/csharp/fundamentals/coding-style/identifier-names

Introducción a .NET

CLR (Common language runtime)

Entorno de ejecucion para .NET. En tiempo de ejecucion el compilador de CLR convierte el codigo CIL en codigo nativo para el SO. Facilita la integración entre lenguajes.

CLR es la MV en la que se ejecutan nuestras apps. CLR se hizo para tener una capa de abstraccion entre las propias apps y el SO donde se ejecutaban.

El CLI se puede ejecutar en otros SO. El CLR se ejecuta solo en Windows.

Read More

.NET launchsettings vs appsettings

launchSettings

NO se despliega. Afecta a tu entorno local.

En él establecemos los perfiles con los que ejecutaremos nuestro proyecto y se usa para definir la variable ASPNETCORE_ENVIRONMENT. También permite establecer la url y puertos de ejecución.

{
  "profiles": {
    "my.project.namespace": {
      "commandName": "Project",
      "launchBrowser": false,
      "applicationUrl": "https://localhost:5011;http://localhost:5010",
	  "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development"
      }
    }
  }
}

ASPNETCORE_ENVIRONMENT Indica el entorno - soporta los siguientes valores

  • Development
  • Staging
  • Production valor por defecto si se omite el valor

appsettings

Se utiliza para almacenar la configuración de la aplicación como por ej. cadenas de conexión de BBDD.
Se utiliza tanto en entornos de desarrollo como de producción.

{
	"LocalDirectory": "/opt/data-download",
	"MyServiceConfig": {
		"Uri": "http://localhost:8800",
		"Endpoint": "/some-endpoint",
		"Timeout": 30
	}
}

appsettings.{Environment}.json

Son archivos adicionales opcionales, donde {Environment} corresponde al valor de la variable ASPNETCORE_ENVIRONMENT del launchSettings.json.

El orden de carga es:

  1. appsettings.json
  2. appsettings.{Environment}.json (si existe)
  3. launchSettings.json

Hay varias maneras de leer la configuración.

leer de la raiz

Las que se encuentran en la raiz las podemos leer inyectando la config en la clase.

public class MyService(IConfiguration _config) : IMyService
{
	public void MyMethod()
	{
		var localDirectory = _config["LocalDirectory"];
	}
}

leer de una clase de config custom

por un lado tenemos la clase de config

public class MyServiceConfig
{
	public const string Section = "MyServiceConfig";

	public string Uri { get; set; }
	public string Endpoint { get; set; }
	public int Timeout { get; set; }
}

y por otro lado la inyectamos

public class MyService(MyServiceConfig _config) : IMyService
{
	public void MyMethod()
	{
		var uri = _config.Uri;
		var endpoint = _config.Endpoint;
		var timeout = _config.Timeout;
	}
}

la tendremos que poner también en el Startup

public void ConfigureServices(IServiceCollection services)
{
	// ...
	services.Configure<MyServiceConfig>(config.GetSection(MyServiceConfig.Section));
	// ...
}

Buenas prácticas

Usar appsettings.json como base y en cada appsettings.{Environment].json poner solo los datos que cambien.

No poner secretos en appsettings.json ni sus variantes:

  • En local (para development) utilizar User secrets
  • Para los entornos con Kubernetes inyectar secretos via ConfigMaps o Secrets

Reference(s)

https://learn.microsoft.com/es-es/aspnet/core/fundamentals/environments?view=aspnetcore-8.0
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/configuration/?view=aspnetcore-8.0

.NET Middleware

Middleware is software assembled into a pipeline to handle request and responses. Each component:

  • Chooses if he passes the request to the next component.
  • Can perform work before and after the next component.

Common use cases:

  • Logging
  • Authentication & Authorization
  • Request/Response processing
  • Caching
  • Error handling

Implementation w. DI

For more details check this project

We create our own custom middleware with an internal dependency, which is the class it’s going to process before/after our request.

// interface and its class, which do the processing
public interface IMiddlewareService
{
	Task ProcessRequest(string request);
}

// interface implementation
public class MiddlewareService : IMiddlewareService
{
	public async Task ProcessRequest(string request)
	{
		// do something with the request
		Console.WriteLine(request);
	}
}

Then we have our middleware class as such

public class CustomMiddleware(IMiddlewareService _service) : IMiddleware
{
	public async Task InvokeAsync(HttpContext context, RequestDelegate next)
	{
		// custom logic to be executed BEFORE next middleware
		await _service.ProcessRequest("middleware processing request");
		await next(context);
		// custom logic to be executed AFTER next middleware
	}
}

Declaration inside Startup.cs

public void ConfigureServices(IServiceCollection services)
{
	// ... whatever

	// register both: our custom middleware, and its internal dependency
	services.AddTransient<IMiddlewareService, MiddlewareService>();
	services.AddTransient<CustomMiddleware>();
	
	// ... whatever else
}
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
	// ... whatever

	// declaration to use middleware
	app.UseMiddleware<CustomMiddleware>();
	// if we use this, the middleware declaration has to be BEFORE or else it won't work
	app.UseEndpoints(endpoints => 
	{
		endpoints.MapControllers();
	})

	// ... whatever else
}

Reference(s)

https://medium.com/@dushyanthak/best-practices-for-writing-custom-middlewares-in-asp-net-core-97b58c50cf9c
https://learn.microsoft.com/es-es/aspnet/core/fundamentals/middleware/write?view=aspnetcore-8.0
https://sardarmudassaralikhan.medium.com/custom-middleware-in-asp-net-core-web-api-70c2ffbbc095

Test APIs with Postman - Scripting

We can set pre-request scripts (run before the request) & tests (after execution) at several levels:

  • Collection
  • Folder
  • Request

Snippets

Inside pre-request Script and Tests we have a SNIPPETS column with templates we may use for our code.

Get / Set variables

console.log("Hello world");

// work with local vars
let urlVar = pm.variables.get("protocol");
console.log("value for protocol: " + urlVar);

pm.variables.set("protocol", "http");
console.log(pm.variables.get("protocol"));  

// work with global vars
let globalVar = pm.globals.get("env");
console.log(globalVar);

Read More

Test APIs with Postman - GUI

To test postman I use https://reqres.in. You can use it to learn how to write Postman tests.

Collections

Collections have metadata which you can set up. This includes:

  • Authorization so you don’t need to set it for every single request.
  • Variables, for variables only for this collection (so you don’t use environment vars).
  • Collection tests.

Read More

Error al crear imagenes en local (Minikube)

Hay un problema al intentar correr una imagen tuya propia en local con minikube. Minikube tiene su propio repositorio de imagenes y si no encuentra una imagen alli, la intenta descargar siempre del repositorio.

Solución

Set imagePullPolicy: IfNotPresent

Run the following command:

eval $(minikube docker-env)
# build again the image
# try to run it again