(este post es una explicación de la teoría. dejo aquí otro post con el detalle práctico de como usar la temperature, top_k y top_p a efectos prácticos)

Explicación más en detalle de cómo obtienen los LLMs las probabilidades para generar el siguiente token

Sampling

En cada posición el modelo tiene una bolsa con miles de tokens posibles. Sampling es el proceso de no coger siempre el token más probable, si no de orientarlo a coger respuestas con determinadas características.

Si el modelo escogiera siempre el token con más probabilidades, obtendríamos respuestas aburridas y repetitivas.

Logits

Para generar el siguiente token, una red neuronal calcula primero los vectores de logits, donde cada logit corresponde a un valor posible. El tamaño de estos vectores de logits es tan grande como el vocabulario completo del modelo.

(representación de vectores de logits)

flowchart LR
	N1["What's your favorite color?"]:::note --> Z
	Z --> A1 --> A
	Z --> B1 --> B
	Z --> C1 --> C
	Z --> D1 --> D
	
	Z["Neural network"]
    A["a"]
    A1["(-0.5)"]
    B["green"]
    B1["(0.7)"]
    C["red"]
    C1["(0.5)"]
    D["the"]
    D1["(-1.2)"]
    
    classDef note fill:none,stroke:none,color:#777;    

Los logits NO representan probabilidades ya que no suman 1 y pueden incluso ser negativos (la probabilidades no pueden). Para convertir logits a probabilidades se usa una Softmax layer

Temperature

La temperatura es una constante que se aplica a los logits antes de la transformación de la Softmax layer. Se usa para ajustar la creatividad del modelo y redistribuir la probabilidad de los valores. Una temperatura más alta hace que el modelo sea más creativo ya que aumenta las posibilidades de elegir tokens menos probables.

xychart-beta
  title "Temperatura vs Probabilidad"
  x-axis "Temperatura (T)" [0.1, 0.2, 0.5, 1, 2, 5]
  y-axis "Probabilidad" 0 --> 1
  line "P(token1)" [0.9999546, 0.9933071, 0.8807971, 0.7310586, 0.6224593, 0.5498340]
  line "P(token2)" [0.0000454, 0.0066929, 0.1192029, 0.2689414, 0.3775407, 0.4501660]

Ejemplos de temperaturas:

• Low (0.2-0.3): El modelo es cauto y elige las palabras más probables. Output factual y predecible.
• Medium (0.5-0.7): Un mix de confiabilidad y engagement
• High (0.9-1.0): Toma riesgos y es impredecible

Read More

Controller test example

this includes how to mock a request’s header - but it’s a bogus test. this only shows how to use XUnit and its structure.

public class XXXControllerTest
{
	// all mocks and stubs
	private Mock<IXXXService> _serviceMock;
	
	// controller under test 
	private XXXController _controller;
	
	public XXXControllerTest() 
	{
		_serviceMock = new Mock<IXXXService>();
		_controller = new XXXController(_serviceMock.Object);
	}
	
	[Fact]
	public async Task CallXXX_ShouldCall_Service()
	{
	// ARRANGE
	
	// mock header
	var httpContext = new DefaultHttpContext();
	httpContext.Request.Headers["someHeader"] = "my-mocked-value";
	_controller.ControllerContext = new ControllerContext
	{
		HttpContext = httpContext
	};
	
	// mock request
	string mockedValue = "someInputValueTo_serviceMock";
	string mockedResponse = "someResponseValueFrom_serviceMock";
	_serviceMock.Setup(mock => mock.SomeMethodCall(mockedValue)).ReturnsAsync(mockedResponse);
	
	// ACT
	var response = await _controller.CallSomething(mockedValue) as OkObjectResult;
	
	// ASSERT
	response.Should().NotBeNull();
	response.StatusCode.Should().Be(200);
	response.Value.Should().BeEquivalentTo(mockedResponse);
	}
}

Basic service test example

public class XXXServiceTest
{
	// all mocks and stubs
	private Mock<IXXXDependency> _dependency;

	// service under test
	private XXXService _serviceMock;

	public XXXServiceTest()
	{
		_dependency = new Mock<IXXXDependency>();
		_serviceMock = new XXXService(_dependency.Object);
	}

	[Fact]
	public async Task ProcessXXX_CaseXXX_ShouldReturnOkay()
	{
		// ARRANGE
		string paramX = "something";
		string responseX = "some response";
		_serviceMock.Setup(mock => mock.SomeMethodCall(paramX)).ReturnsAsync(responseX);
		
		// ACT
		var result = await _service.ProcessXXX(paramX);
	
		// ASSERT
		result.status.Should().NotBeNull();
		// ... assert whatever
	}
}

If we want to use a method that’s marked as async inside a lambda expression, we have to split it in 2 steps:

• task declaration
• (async/await) task execution

example 1

var adminUserTask = users
	.Where(user => "admin".Equals(user.type.ToLower()))
	.Select(async user => { return await ProcessAdmin(user);});
List<UserResults> results = (await Task.WhenAll(adminUserTask)).ToList();

example 2

// task declaration
var mapTask = animals.Select(Map).ToList();

// task execution
var animalsMapped = (await Task.WhenAll(mapTask)).ToList();

// mapping method
private async Task<Animal> Map(Animal animal)
{
	// ... do whatever mapping is needed
}

example 3

private async Task<List<AnimalDTO>> MapAnimalsToDto(List<Animal> animals)
{
	var dtos = await Task.WhenAll(animals.Select(a => MapSingleAnimal(a)));
	return dtos.ToList();
}

// method signature
private async Task<AnimalDTO> MapSingleAnimal(Animal animal);

The following is an example of the .sh scripts I use to forward and debug pods or features.

#!/bin/bash
# to use: sh portforward_microservice_xxx.sh env-dev | env-pre | env-pro
# it accepts additional params such as no-connectors to ignore some port forwards
# 	no-connectors - ignores forwards to XXX
args=("$@")
echo "INFO: using namespace ${args[0]}"

# only needed if we have several clusters for each env
if [ $1 == "env-pre" ]; then
	kubectl config use-context context-for-pre
elif [ $1 == "env-pro" ]; then
	kubectl config use-context context-for-pro
else	
	# default value - always DEV
	kubectl config use-context context-for-dev
fi

# add here new variables or cases to omit
no_connectors=false
for arg in "$@"; do
	case $arg in
		no-connectors)
			no_connectors=true
			shift
			;;	
		*)
			shift
			;;
	esac
done

if [ "$no_connectors" = false ]; then
	kubectl port-forward -n ${args[0]} svc/connector1 9210:9210 &
	kubectl port-forward -n ${args[0]} svc/connector2 8000:8000 &
else
	echo "INFO: ignoring connectors"
fi

# common pods we always need to call
kubectl port-forward -n ${args[0]} svc/service1 5050 &
kubectl port-forward -n ${args[0]} svc/service2 5060

(TODO: add link -> working code inside this project)

Let’s set a case where we have the following User class where we want to soft delete it, as we want to keep deleted records.

public class User
{
	public int Id { get; set; }
	public string Name { get; set; }
	public bool Active { get; set; }
}

In many cases we don’t care about “deleted” records so most times we will filter out deleted records like this

// DON'T DO THIS
public async Task<List<User>> GetUsers()
{
	return await _context.Users.Where(user => user.Active).ToList();
}

Instead of always doing this, which is too verbose, we may use Global Query Filters. This way we apply the filter globally.

public class ApDbContext : DbContext
{
	// ... more code
	
	protected override void OnModelCreating(ModelBuilder modelBuilder)
	{
		modelBuilder.Entity<User>().HasQueryFilter(user => user.Active);
	}
}

From now on, everytime we need to retrieve something from the User table, it will automatically filter out the deleted records.

Read More

Para documentar metodos en .NET es comun utiliar comentarios XML, mediante los cuales se puede generar documentacion externa usando herramientas como DocFX o Sandcastle.
Esto permite que tu API o biblioteca tenga una descripcion completa de cada método.

Ejemplo de comentario

public class ProductsController : ControllerBase
{
	/// <summary>
	/// Get a specific product by its ID.
	/// </summary>
	/// <param name="id">The ID from the product to retrieve</param>
	/// <returns>Returns the product if it's found. Otherwise it returns HTTP 404.</return>
	/// <response code="200">If the product is found.</response>
	/// <response code="404">If the product is not found.</response>
	[HttpGet("{id}")]
	public IActionResult GetProductById(int id)
	{
		var product = // get product from a service
		if(product is null)
		{
			return NotFound();
		}
		
		return Ok(product);
	}
}

For many apps, a basic health probe configuration that reports the app’s availability to process request is sufficient to discover the status of the app

At Startup.cs we add the following

public void ConfigureServices(IServiceCollection services)
{
	// ... other configuration
	services.AddScoped<IUserService, UserService>();

	// add health checks
	services.AddHealthChecks();
}

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
	// ... other configuration

	// map health checks to an specific endpoint
	app.UseHealthChecks("/beat");
}

This endpoint will now be available at the service’s root. If this publish f.e. at port 5000, we can now call the following endpoint

http://localhost:5000/beat

// response
status 200, Healthy

The problem with this basic check is that if something fails inside the constructor of a controller or a service, this still returns status 200, Healthy.

Read More

Ambas son dos maneras de reutilizar código para crear nuevas clases

Herencia

Permite que una subclase herede propiedades y métodos de su superclase o clase base. Facilita la reutilización de código y creación de relaciones jerárquicas entre clases.

Sin embargo puede llevar a estructuras de clases rígidas y complejas, y a problemas de acoplamiento.

Composición

En lugar de heredad de una clase base, la clase se “compone” de otras clases incluyendo instancias de otras clases como campos.

Esto promueve un diseño más modular y flexible ya que permite cambiar el comportamiento en tiempo de ejecución.

Depende del problema específico pero la composición suele ser preferida por su flexibilidad y capacidad para evitar problemas comunes de herencia.

Read More

These principles establish practices that helps maintain and extend software as it grows.

S - single responsibility
O - open/closed
L - liskov substitution
I - interface segregation
D - dependency inversion

Single responsibility

A class should have only one job - it should have only one reason to change

Read More

(This is an implementation example. For an explanation on this, please check my other post: SQL Triggers & Stored Procedures)

Function example

-- example of function that triggers when an entry is inserted into a table
--   and manages this data inserting data as needed in another table
CREATE OR REPLACE FUNCTION your_schema.my_function_name(arg1 character varying, data character varying)
	RETURNS character varying
	LANGUAGE plpgsql
AS $function$
BEGIN

-- check if already exists in the other table 
IF
	(SELECT COUNT(*) FROM your_schema.other_table WHERE name=arg1) > 0) THEN
		RETURN 'Error: 1210. data already exists';
END IF;

-- insert and manage data
INSERT INTO your_schema.other_table (name, data) VALUES (arg1, data);
RETURN 'Success: 1200';

END $function$;

How to Debug in DBeaver

There are two options, logs or break the function with an exception.

If logs are enough, you just write the message to output per console.

RAISE NOTICE 'this is null';

To see logs in DBeaver click here. Then you may execute the function to see the logs.

If you want to break the function runtime with an exception, you write the following instead

RAISE EXCEPTION SQLSTATE '90001' USING MESSAGE = 'error. this already exists';

SQL View

A view in SQL is essentially a virtual table. It doesn’t store data physically. Instead it presents data from one or more underlying tables through a predefined SQL query. Think of it as a saved query that you can treat like a table.

• Views don’t hold data themselves. When you query a view, the database executes the underlying query to fetch data in real-time.
• Views can simplify complex queries by encapsulating them. Instead of writing a complex JOIN or subquery each time, you select from the view.
• Views can restrict user access to specific rows or columns, enhancing security by exposing only necessary data.
• Since views are generated on the fly, they always reflect the current state.

CREATE OR REPLACE VIEW active_customers AS
SELECT customer_id, name, email
FROM customers
WHERE status = 'active';

When to use a view

• Use it to simplify complex queries that you use frequently and you need the most current data every time.
• Restrict user access to specific data by exposing only certain columns or rows through a view

Materialized View

A materialized view is like a regular view, but it stores the query result’s phisically and it doesn’t involve executing the underlying query each time.

• Since data is precomputed and stored, querying a materialized view is faster for complex queries over large datasets.
• Because of this, data in a materialized view can become outdated and needs to be refreshed periodically.

CREATE OR REPLACE MATERIALIZED VIEW sales_summary AS
SELECT product_id, SUM(quantity) AS total_quantity
FROM sales
GROUP BY product_id;

When to use a materialized view

• Is ideal for speeding up complex queries that are resource-intensive and slow to execute.
• Suitable for scenarios where data doesn’t change frequently and fast read performance is needed.
• You need to tolerate data that’s not always up-to-date