
## Introduction

[Interep](https://interep.link) is a service that ties web2 identities to a reputation tier, and each reputation tier on a given web2 platform(Github, Twitter, Reddit) is a [Semaphore Group](https://semaphore.appliedzkp.org/docs/glossary#semaphore-group) which allows the user to anonymously prove that they are a part of a given "reputation set" on Interep. 

By restricting access to certain resources to peers which belong to the higher reputation groups, we obtain sybil-resistance for those resources. However, this does increase the barrier of entry. For example, a new user on Github may not have the stars and followers required for a good reputation, and therefore, may miss out on membership to the "reputation set". In support of Interep, the barrier of entry is much lower than that of a financial barrier of entry, i.e Financial stake. Financial stake is a predominant solution for sybil-resistance, but it allows actors with large coffers to create sybil identities.

[RLN](https://rfc.vac.dev/spec/32/) (Rate Limiting Nullifier) is a modification of [Semaphore](https://semaphore.appliedzkp.org/docs/introduction) which effectively allows peers to derive the secret share of a peer that spams a network. This enables anonymous rate-limiting in a decentralized scenario, but can be applied in conventional situations as well. 

An implementation of RLN is found in [Waku](https://rfc.vac.dev/spec/17), which makes use of a smart contract to handle changes to a membership set. When a peer wishes to send messages on a spam-protected content topic, they simply need to stake a nominal amount, along with their [identity commitment](https://semaphore.appliedzkp.org/docs/glossary#identity-commitment), and relayers will validate messages sent by the peer. However, if the peer is malicious, and spams the network, their secret share is derived, and their financial stake is withdrawn by the relayer which derived their secret share.

## How Interep fits in with RLN

As mentioned, to be a part of an RLN membership set in a decentralized scenario, one needs to financially stake. With Interep, we can reduce the barrier of entry by allowing only peers which belong to a medium/high reputation group to join the RLN membership set, without staking.

When this "socially staked" peer participates in spam, their secret share is revealed, and they are removed from the membership set, as usual. A third-party on-chain reputation service could then reduce the reputation for the slashed peer.

## Work Done

So far, the RLN contracts have been updated and currently reside in [rln-interep-contract](https://github.com/vacp2p/rln-interep-contract). It is possible to register your membership on Interep, and use the same credentials for rate-limiting via RLN. A demo video is accessible [here](https://drive.google.com/file/d/1jE4XxByugVtQA2McozcaXIb_6Y3bkYAM/view?usp=share_link).

## Future Work

Currently, Interep's reputation service is centralized, i.e For a user to prove that they have a given reputation, they need to send an auth token to Interep's service, through which they are added to the appropriate reputation set. There may be some effort done to decentralize this service for censorship resistance.

As mentioned previously, a third-party on-chain reputation service should reduce the reputation for a slashed peer. However, that service has not been integrated so far, and it can be done in the future.

Some careful consideration must be taken care when dealing with web2 providers, as their metrics for a non-sybil account may change over time, and a user's reputation may change with time as well. For increased security, we may attach a timer for each registration to the membership set via Interep so that they can renew their membership with an additional proof that they still belong to the same reputation set.

There are also on-chain Interep groups available through [POAPs](https://poap.xyz/), but they have not been integrated with this PoC. For example, If a DAO wishes to participate in rate-limiting, and should allow only its members to do so, they can issue a POAP to all the members, and the members may then use this to generate a proof that they belong to the DAO. The service can then filter traffic appropriately.

## References
1. [Interep](https://interep.link)
2. [Semaphore](https://semaphore.appliedzkp.org/docs)
3. [RLN Spec](https://rfc.vac.dev/spec/32)
4. [Waku-RLN-Relay Spec](https://rfc.vac.dev/spec/17)
5. [RLN Interep Contract](https://github.com/vacp2p/rln-interep-contract)
6. [POAP](https://poap.xyz/)

Thanks for reading!

Anonymous rate-limiting with web2 reputation?

Using Interep for Sybil Resistant RLN



## Introduction

While working at [Status](https://status.im) I have come to realize that the path to decentralization is only brought forward by making any network, more accessible. This would require minimal bandwidth usage as well as compute utilization. 

In discussions and debates surrounding the optimal type of zk-EVM, a significant oversight often arises with regards to the perpetuation of critical security vulnerabilities within the existing tooling that accompanies the pursuit of "compatibility." Although creating an entirely new ecosystem of tooling for a novel high-level language that compiles to bytecode is undeniably challenging, it becomes necessary to broaden the scope of decentralization, a fundamental principle in the space.

## How type 4 zk-evm's differ from the rest

Type 4 zk-evm's take one major requirement out of the game, "compatibility". With this, they are able to make use of specialized languages, which are optimized to be faster at proving, which have the following advantages -

1. more efficient transaction validation
2. reduced risk of delays due to block processing
3. throughput of the network

which ultimately all lead to better user experience. If the user experience is great, then it is almost guaranteed that the network will attract more users and node operators over time, increasing the decentralization.

## How some disadvantages are truly the strengths

The main disadvantages of type-4 systems that come to mind are -

1. lacking compatibility with existing ethereum applications
2. lack of capability to use shared infra
3. lack of tooling

There are ways to convert these disadvantages into strengths, as explained -

1. lacking compatibility with existing ethereum applications
    > Compatibility is not a concern for teams that would be zero-ing in on a specific chain for their use-case. It is worth the engineering effort to rewrite parts of a protocol to support a chain which will bring in more value. However, teams generally do not have this bandwidth, and efforts must be taken to ease the process for them. There are a handful of applications which use inline bytecode in their contracts, which would most likely not be necessary for type 4 zk-evm's, making the engineer's life easier in the long run, in terms of maintenance of such a codebase.

2. lack of capability to use shared infra
    >  Shared infrastructure does offer advantages for Type-1 and Type-2 zk-EVMs, but it also brings along the risk of inheriting security flaws from the existing infrastructure. However, when specialized infrastructure is employed, although there is still a possibility of security issues, they can be audited by multiple teams, thereby reducing the overall risk exposure. By subjecting the specialized infrastructure to rigorous security audits, the chances of vulnerabilities can be minimized, providing increased confidence in the security of the zk-EVM system.

3. lack of tooling
    > The absence of tooling for a Type 4 system can be mitigated by implementing bounty programs, which attract a new set of engineers to contribute and harness the benefits of available open-source resources. In contrast, other types of zk-EVM systems often face challenges in codebase retention and maintenance due to developers frequently transitioning between different systems.

## What's it looking like for type-4 systems now?

[zkSync Era](https://zksync.io/) is a type-4 zk-evm with solidity support. A [dune dashboard](https://dune.com/unionepro/zksync-era-stats) shows that it has 633M USD TVL as on this date. While most of the addresses belong to airdrop farmers, it is only after the airdrop that we will see actual use of this network. Another [dashboard](https://app.zettablock.com/community/dashboards/da-17f302ae-7f16-417d-a2eb-9cb611d7c361) shows that the median transaction fee is 0.29 USD, which is lower than other zk-evm networks. Ultimately, this will drive user adoption, and with better tooling, is a very viable competitor in the space.

Thanks for reading!



Discussion about why type 4 systems may be underrated

Type 4 ZK-EVM's are slept on




## Introduction
The following post provides an overview of the Waku determinsitic pubsub topic sharding method, which is based on the [35/WAKU2-NOISE](https://rfc.vac.dev/spec/35/) and [23/WAKU2-TOPICS](https://rfc.vac.dev/spec/23/#23waku2-topics) RFCs.

## How does it work?

The Waku pubsub topic sharding method is based on the use of a shared secret key derived from a Diffie-Hellman key exchange, and a deterministic hash function. 
The method is described as follows:

1. The two parties, Alice and Bob, establish a shared secret key using a Diffie-Hellman key exchange.

2. The shared secret key is used as an input to a deterministic hash function, such as SHA256, to generate a new topic for the next message.

    $$
    pubsub\_topic = sha256(shared\_sk  \ mod \  privacy\_parameter)
    $$

3. For each subsequent message, the shared secret key (which has been recomputed) is used as the input to the hash function again to generate a new topic.

4. (optional) To ensure the topic is unique, a nonce is concatenated to the shared secret key before hashing.

    $$
    pubsub\_topic = sha256((shared\_sk  \ mod \  privacy\_parameter) ⌢ nonce)
    $$

5. (optional) To ensure the topic is different for each message, a counter is concatenated to the shared secret key before hashing.

    $$
    pubsub\_topic = sha256((shared\_sk ⌢ ctr)  \ mod \  privacy\_parameter)
    $$
    
6. (optional) To reduce computational overhead of calculating a new pubsub topic for each message, a time window can be agreed upon between Alice and Bob. 
  For example, a 24hr window can be negotiated if it is within the constraints that both Alice and Bob agree to.

Therefore, the pubsub topic is calculated in the following manner

$$
pubsub\_topic = sha256((shared\_sk⌢ timestamp)  \ mod \  privacy\_parameter) 
$$

Note that this approach would require both Alice and Bob to keep track of the shared secret key used at the start of the negotiation to derive the next pubsub topic. 
This cache can be erased when the negotiation process restarts at the end of the window.


The `privacy_parameter` can be set by the peers, and must be agreed on beforehand between Alice and Bob. 
The lower the `privacy_parameter` is, the set of values for the pubsub topic would be fewer and it offers k-anonymity based privacy benefits. 
The higher the `privacy_parameter` is, the set of values for the pubsub topic 'B' would be larger, therefore, performance is increased at the cost of privacy.

It's important to note that the shared secret key changes with each message sent, as per Noise processing rules. 
This allows updates to the shared secret key by hashing the result of an ephemeral-ephemeral Diffie-Hellman exchange every 1-RTT communication. Therefore, even if privacy guarantees are lost with one message, they can be regained with the next message, if Alice signals to Bob (or vice-versa), to change the privacy_parameter

One can argue that this approach may lead to failure in forming a mesh, but with the assumption that: 
- the number of community-run nodes are high, and they participate in meshing
- peers are incentivised to relay messages on a pubsub topic (incentivisation is yet to be solved)

If any of these assumptions are invalid, the peers can resort to using a lower `privacy_parameter` which would result in a commonly used pubsub topic, and piggyback on other peers that have a vested interest in that pubsub topic.

## Security Considerations
The Waku pubsub topic sharding method has several potential security considerations that must be taken into account, including:

1. The security of the shared secret key depends on the security of the Diffie-Hellman key exchange, which can be vulnerable to various attacks if not implemented correctly.

2. The privacy benefits of k-anonymity depend on the value of the privacy parameter. A low value of the privacy parameter may provide stronger privacy guarantees, but at the cost of lower performance. A high value of the privacy parameter may provide better performance, but at the cost of weaker privacy guarantees.

3. An attacker who is able to obtain the shared secret key or the privacy parameter may be able to determine the topic of a message and potentially intercept it.

4. The technique of updating the shared secret key with each message can be useful in protecting against eavesdropping, but it also requires additional computation and communication overhead.

5. The technique of updating the shared secret key with each message also requires a secure signaling mechanism to signal the privacy parameter between Alice and Bob.

Overall, it is important to carefully consider the trade-offs between performance and privacy when implementing the Waku pubsub topic sharding method, and to ensure that the key exchange and signaling mechanisms are implemented securely.

## The big problem with deterministic sharding

Forming a mesh with peers constantly changing pubsub topics is a challenge, but it can be solved by incentivizing peers to relay messages on a pubsub topic, via private settlement.

## Conclusion

The Waku pubsub topic sharding method is a promising approach to improving the bandwidth efficiency of Waku, but it also has several potential security considerations that must be taken into account. It is important to carefully consider the trade-offs between performance and privacy when implementing the Waku pubsub topic sharding method, and to ensure that the key exchange and signaling mechanisms are implemented securely.


Thanks for reading!

Improve bandwidth efficiency of Waku by using deterministic sharding

Scale Waku with deterministic sharding


## Introduction

Censorship resistance is enabled in large scale peer-to-peer networks like Ethereum or Bitcoin, since even one node could allow a message to be broadcasted to the rest of the network. However, when a large scale sanction on a particular service is made, as with [Tornado Cash](https://www.coindesk.com/policy/2022/08/08/crypto-mixing-service-tornado-cash-blacklisted-by-us-treasury/), then the entire community seems to rally with it, thereby, censoring the acts of the individuals associated with Tornado Cash. While this may be ethically sound, this has now set a precedent for cancel-culture, or SEC-induced sanctions to be viable for *any* decentralized protocol in the future. While others claim that this is also thwarted by *forks* of the original protocol, and that the SEC or any related authority cannot efficiently place sanctions on all look-alikes, due to the operational overhead, the quality of these look-alikes is often ignored, and therefore results in sub-optimal outcomes, which is worse for censorship resistance.

## Societal Good vs Network Good

To truly prevent censorship, we must also acknowledge that with censorship resistance, we allow anti-social individuals to actively participate in the network. While they may not be a bad actor to the network, i.e, they may participate in the network with 99.9%+ uptime, have 0 equivocations, etc, but, they may be a bad actor in *society*. The fact that someone is a bad actor in *society* often changes with time, whereas a bad actor in the network, is a bad actor in the network for all time to come. This fundamental difference is what causes the pushback from those who are in control, in fear of the chaos that could be caused by societal bad actors. In order to have a truly decentralized network without censorship, it is imperative to include all good actors of the network, regardless of any other aspect which may label them as *good* or *bad*.

This does not mean that networks which emphasize societal good-ness shouldn't exist. Such networks could be useful in generating a *reputation* for arbitrarily good actors, which could power a whole suite of protocols which rely on an individual's reputation (Example - [Augur](https://augur.net/), [oracles](https://reputation.link/), etc). The individual's reputation can be built over a long period of time, but slashed almost immediately based on their [actions](https://thousand-validators.kusama.network/#/getting-started). This could prevent someone from acting adversely, at the cost of censoring their true action.

The game-theoretic aspect of censorship resistance is indeed unique, where a network does not want it's peers to Dox themselves to hold them accountable, nor does it want to enable societal bad actors from participating in the network. This is partially solved by having different networks with different use-cases, as mentioned above.

## Barrier of Entry

A [paper](https://theoryofcomputing.org/articles/v003a001/v003a001.pdf) published in 2007, mentions censorship resistance of the data stored on a peer-to-peer network, but ignores the participation aspect of nodes on the same network. This also circles back to the different consensus mechanism's that blockchains use, like [PoW](https://www.scs.stanford.edu/17au-cs244b/labs/projects/porat_pratap_shah_adkar.pdf), and [PoS](https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/). An interesting fact is that Proof of Work was used in [email spam prevention](https://medium.com/coinmonks/how-preventing-spam-emails-led-to-proof-of-work-46a294b2435), years before it was actually used on the Bitcoin blockchain. Many have varied opinions on the use of PoW vs PoS, with reasons like energy efficiency, barrier of entry, etc. An analysis of barrier of entry to compare the two is required. 

### PoW

The barrier of entry with Proof of Work systems is dependant on the difficulty of the computation, which increases with the number of blocks produced by the network (trivially). The higher the difficulty to produce the next block is, the more expensive it is to acquire the hardware which can produce the next block quickly and efficiently. This allows only a subset of actual individuals to participate in PoW networks, and facilitated the creation of organizations like [Mining Pools](https://slushpool.com/en/home/). Mining pools account for 1/5th of [Bitcoin's hashrate](https://arcane.no/research/bitcoins-hashrate-is-going-public), which reduces the decentralization of the network, and in turn, censorship resistance.

At the time of writing this article, it costs approximately $2000 to procure a Bitcoin mining rig, which turns a profit only after a year of mining. 

The average salary in India, with a population of over 1.2 Billion individuals, is $380 a month, and $4554 a year. It is impractical for an average person to spend a little under half their yearly net income on a Bitcoin mining rig, with the hope that it is able to keep up with the difficulty with time. There are a lot more costs associated with running a PoW rig, like maintainance, optimal cooling, hardware upgrades, etc. 

Conclusion: Lower income individuals are censored by PoW

### PoS

The barrier of entry with PoS systems is relatively lower in terms of monetary amount over a long period of time. However, the initial investment is large, since the stake is used to secure the network. 

It costs approximately $59,000 to set up an Ethereum validator, and running costs to make sure that it has the least downtime as possible. This is much higher than the initial cost for PoW rigs, but the hardware costs and maintainance are cut down massively. Due to this higher initial cost, this leads to [Staking Pools](https://academy.binance.com/en/glossary/staking-pool) forming, which pool together the stake of multiple users, which, again, reduces decentralization, and in turn, censorship resistance.

Conclusion: Lower income individuals are censored by PoS


### PoW, PoS, and Sybil Resistance

The reason that it is very expensive become a block producer on PoW or PoS based systems, is to prevent [Sybil attacks](https://www.sciencedirect.com/topics/computer-science/sybil-attack). Sybil attacks are caused by the lack of identity on either networks, which again, if identities were a part of these networks, would enable censorship resistance.

This is a very difficult problem to solve, in a way that does not compromise confidentiality, integrity and authenticity of the peers. 


## Conclusion

While censorship-resistance may not be mature in the current state of peer-to-peer networks, there is much research being done in the space, from notable teams who run [Monero](https://www.getmonero.org/), [Secret Network](https://scrt.network), [ZCash](https://z.cash/), and [Status.im](https://status.im/).



Thanks for reading!

Censorship in p2p networks


## Introduction

It's been a while since my last post, and that's because I've been focusing on work, along with finishing up university. I haven't had much time to keep up with developments in the crypto ecosystem. I'm okay with that, because it was just NFTs :P

I am an advocate for KubeMQ, with the ease it provides versus other MQ servers while operating on K8s. However, due to certain reasons I won't get into, we had to migrate our MQ Solution. The question that came up was - How do we do it with zero downtime? This post is precisely about that.

### Step 1: Analysis
As part of the initial analysis, we had to nail down a few details - 

- Ideal MQ server for migration
- Prioritize workloads according to MQ dependency

For the first detail, we looked at many MQ servers, some of which I do have experience with. We ended up going with RabbitMQ, because it offers both ephemeral and persistent queues, along with being highly configurable with few changes to the codebase. Not to mention, the ease of deployment as well.

For the second detail, an ast-parsing script was written that went through all our workloads to check whether they published/subscribed to a channel. With this list of workloads, we then began to prioritize them according to mission importance. There are other ways to find out which workloads use the pubsub feature, for example, it is possible to get a list of clients from KubeMQ's REST API.

### Step 2: Work out RabbitMQ's configuration

As part of deploying RabbitMQ, we needed to ensure that we used an appropriate configuration, for things like user management etc. Since we like looking at dashboards, we also had to enable metrics. Metrics also gives us a look into the performance of RabbitMQ vs KubeMQ.

### Step 3: Deploy RabbitMQ

Deploying RabbitMQ was fairly easy, as we use helm charts for majority of third party services. We used Terraform for this, obviously :)

### Step 4: Update client libraries to use RabbitMQ

None of our workloads directly make use of KubeMQ's client libraries, they make use of a thin wrapper we wrote over the KubeMQ implementation. This way, we can change the underlying MQ server without having any changes client side :)

We have 2 packages for the MQ implementation, one for ephemeral and one for persistent queuing. Both were updated to use RabbitMQ. A new version of both these packages were created.


### Step 5: Testing the waters

To test if - 

- RabbitMQ was configured correctly
- Client libraries were OK for prod use

We bumped the versions of the non-critical workloads to use the new pubsub packages, and triggered a deploy of them as well. After ironing out a couple bugs that occurred, we were ready to migrate our mission critical workloads to use those packages too.

### Step 6: Farewell, KubeMQ

After our main workloads started to use RabbitMQ, and we were satisfied with the performance, it was time to deprecate KubeMQ in our org. To do this, we simply deleted the `kubemq` namespace from our Terraform config. Any of the services that prevented the namespace deletion were also cleaned up forcefully.

All this took place on a development cluster, and moving this to a staging/production cluster is as easy as 1-2-3 (best case, lol)

### Conclusion

I haven't done an MQ migration before, and thought to share my workflow behind it. I would appreciate any feedback you have/tips as well (HMU @ hello@rymnc.com)

Thanks for reading!


How we migrated between MQ servers with 0 downtime

Migrating Message Queues


## Introduction

To locally setup InfluxDB every other day with a "docker run" or docker-compose becomes cumbersome. All additional buckets, users, that need to be created can't be declaratively done. A workaround would be to just write init scripts that took care of it, but they are susceptible to API changes and require maintenance.

An alternative to doing this would be to set it up with Terraform. This would allow you to create buckets, users, tokens, with code, which can be shared with your fellow devs as well!

*Note: The following is with reference to v2+ of InfluxDB*

## Existing Terraform providers for InfluxDB

Luckily, Terraform providers for InfluxDB exist, written and open-sourced by the folks over at [Lancey Energy](https://lancey.fr). You can find the provider documentation [here](https://registry.terraform.io/namespaces/lancey-energy-storage). You may notice that there are 2 providers, one called `influxdb-v2` and one called `influxdb-v2-onboarding`. This is with reference to the fact that running InfluxDB is a 2-step procedure, first being to setup the db, and the next to actually run the container. This process has been abstracted away by these 2 modules.

### Step 1: Initialize your Environment

Since this setup is primarily be targeted towards local development, we will not require any TLS certificates, or remote state.

```bash
mkdir my-influx-config
``` 

```bash
cd my-influx-config && terraform init
```


Create the following files 
```bash
touch main.tf variables.tf outputs.tf versions.tf providers.tf buckets.tf
```

Edit the `versions.tf` file
```hcl
terraform {
  required_version = "1.0.4"
  required_providers {
    docker = {
      source  = "kreuzwerker/docker"
      version = "2.15.0"
    }
    influxdb-v2-onboarding = {
      source  = "lancey-energy-storage/influxdb-v2-onboarding"
      version = "0.4.1"
    }
    influxdb-v2 = {
      source  = "lancey-energy-storage/influxdb-v2"
      version = "0.4.1"
    }
  }
}
```


Re-Init your workspace
```bash
terraform init
```

Copy the `variables.tf` and `providers.tf` from this [repo](https://github.com/rymnc/terraform-influx)


### Step 2: Defining the Docker Container Spec

Edit your `main.tf` file to contain definitions for the InfluxDB version number, volumes and the container itself
```hcl
/* Docker Image for InfluxDB v2 */
resource "docker_image" "influxdb_image" {
  name = "influxdb:${var.influxdb_version}"
}

/* Docker Volume for InfluxDB v2 */
resource "docker_volume" "influxdb_volume" {
  name = "influxdb-data"
}

/* Docker Container for InfluxDB v2 */
resource "docker_container" "influxdb_container" {
  name         = "influxdb"
  image        = docker_image.influxdb_image.latest
  network_mode = "host"
  volumes {
    volume_name    = docker_volume.influxdb_volume.name
    container_path = "/data"
    host_path = "${path.root}/influxdb-data"
  }
  env = [
    "INFLUXD_BOLT_PATH=/data/bolt",
    "INFLUXD_ENGINE_PATH=/data/engine",
    "INFLUXD_HTTP_IDLE_TIMEOUT=0",
    "INFLUXD_QUERY_MEMORY_BYTES=${var.query_memory_bytes}",
    "INFLUXD_QUERY_QUEUE_SIZE=${var.query_queue_size}",
    "INFLUXD_SESSION_LENGTH=12000",
  ]
  ports {
    external = var.expose_port
    internal = 8086
  }
}
```

What this does is creates a Docker Container with the BoltDB and Engine directories being bound to a Docker Volume which is at the root of the tf workspace. I've increased the session length so that I don't need to re-login a day later. 

### Step 3: Setup InfluxDB

Before we run the apply, we have to write some code which sets up the database for use (with initial username, password, bucket name etc).

You must think that we could have used the "DOCKER_*" env arguments to automate this process, but it does not return the org_id, which we need to create buckets in an organization. To make things easier, I've open sourced the modules.

Add the following to your `main.tf`

```hcl
data "external" "wait" {
  program = ["./wait_for_influx.sh", local.influx_url]

  depends_on = [
    docker_container.influxdb_container
  ]
}

/* Setup process for InfluxDB V2 */
module "setup_influxdb" {
  source = "https://github.com/rymnc/terraform-influx.git//modules/setup"

  init_username = var.init_username
  init_password = var.init_password
  init_bucket   = var.init_bucket
  init_org      = var.init_org

  depends_on = [
    data.external.wait
  ]
  providers = {
    influxdb-v2-onboarding = influxdb-v2-onboarding.onboarding
  }
}
```

Since it takes the container a few seconds to be ready to receive requests, we use an external data source, with a simple bash script that polls a URL using curl. As soon as the response is 200, it exits. The module that initializes InfluxDB has to run only after it is responsive, that's why we have the `depends_on` field in the module definition.

### Step 4: Create the required buckets

Add the following to the `buckets.tf`,

```hcl
module "bucket_test" {
  source = "https://github.com/rymnc/terraform-influx.git//modules/bucket"

  bucket_name        = "test"
  bucket_description = "my super cool test bucket"
  org_id             = module.setup_influxdb.influxdb_details.org_id
  retention_days     = 15
  depends_on = [
    data.external.wait
  ]
}
``` 

This just creates a test bucket. Since it depends on external data, which is available only after InfluxDB is responsive, there wont be any failed requests to the url.

You can create as many buckets as you'd like here, or use a for_each and iterate over a map of buckets defined as a variable.

### Step 5: Launch!

After editing your `terraform.tfvars` file, run
```bash
terraform apply # Note that if you're using podman, sudo will be required
```

Give it a minute to pull the image, and then the InfluxDB instance will be ready for development! You can create access tokens as well, by modularizing the existing provider.

### Step 6: Tear down

To destroy all resources
```bash
terraform destroy
```


## Conclusion

Using Terraform with InfluxDB allows me to have reproducible setups accross different machines, and saves time as well. Hope you liked it!



Running and Managing a Local InfluxDB Instance

It's been a while since my last post, that's mainly due to me being super busy with work, and additional college assignments.

Basically, this is how I've been these past few months -


 <img width="50%" src="/assets/hustle.gif">


## Introduction

I've been using Terraform in production recently, and it gave me a whole new perspective on how infrastructure should be managed. A lot of the learning was done with the help of "Terraform in Action", by Scott Winkler. Here are a few things I've picked up on about Terraform, along with a couple opinionated tips 👀️

### Keep an eye on the number of resources

With Terraform workspaces, it can be easy to just set it up and ignore it throughout the lifetime of the project. However, as your workspace starts to expand, the number of resources can definitely affect your plan/apply time. A workspace should definitely only contain resources that heavily depend on each other.

For example, if you have a Kubernetes Cluster with 5 namespaces, each having their own Deployments, and depend largely on other workloads in the namespace, it would be ideal to split it into another workspace. If you need to share some data like IP addresses, ports, etc, you can do so by using the origin workspace as a data source. A point should be noted however, a change in the origin workspace output does not run a refresh in the target workspace. The target workspace needs to have an additional plan and apply executed to act on the new output. To deal with this, you can build weak dependencies into your CI system to automatically run a refresh on dependent workspaces.

### Make use of modules

Initially, while setting up a few Kubernetes Deployments with Terraform, I didn't mind manually writing the secrets, config, etc each Deployment required, however, as the project grew, it became unsustainable. The labels were janky, and declaring Kubernetes resources on Terraform is a pure nightmare (If you thought YAML was bad).

I decided that most of the boilerplate could be abstracted away into a Kubernetes Deployment module, which could just take in the necessary arguments, like resource limits, configs, secrets, and dependent databases etc. This cut down the LoC by A LOT, and made maintaining our entire set of deployments super easy.

Our logging service, has a known bug - that it cannot figure out the container name from the logging process. A workaround for this, was to inject an ENV variable into the container, called CONTAINER_NAME. Now, if I had to update dozens of YAML manifests with this, I'd have to create a script that did it. With Terraform, all I had to do was edit the deployment module, and all the deployments were updated! Note that it is not a Terraform-specific feature, a similar feat can be achieved with Kustomize or Helm.

However, Kustomize lacks in a very important area. It assumes that the deployment is the same from the last time it was created. This is where Terraform shines, since before any change to infrastructure, it detects drifts, and suggests a course of action, i.e to update the resource according to its spec defined by code.

Kustomize 0 | Terraform 1

### Your CD system will change

While trying to adopt Terraform into our system, significant changes had to be made to our CI/CD workflows. Instead of using a tool like gke-deploy to apply manifests, we had to use the Terraform API. Luckily, Terraform has amazing documentation, which really moved along the update to our pipeline.

Using remote state management is VERY important. Even if it is a small project, if You're using Terraform in your pipeline, you need remote state. We use Terraform Cloud, which handles all our needs adequately.

There are however, a few drawbacks that are noticeable. On the one hand, you can trigger a new apply every time a new image is built, and on the other, you can trigger a new apply after all images related to a commit hash are built. With the former, you can rack up a huge amount of pending plans in your workspace, and if you don't set appropriate timeouts, the pipeline will fail. With the latter, atomicity is lost between image builds. For example, if 5 images are built, and 1 has a bad deploy, the entire apply will fail, and reconciling remote state is a cumbersome process (It takes time).

The first option is ideal, if you split up the workspaces into tight units. This frees up the locks on unrelated/weakly related resources! Life. Hacked.

### Version your modules!

It is important to split your actual infrastructure and Terraform module repositories. Having different git histories for both is cleaner. Whenever a module is declared in the infrastructure repo, make sure that it refers to a fixed hash on the module repo. This way, even if the module is updated, it doesn't break the resources that haven't been updated with it! Think of it just like package management.

Versioning modules is definitely something that should be done from Day 0. After all, you don't want to update a module a year later and then notice your services don't talk to each other anymore! (For those of you menaces that don't read the Terraform plan output😝️)

### Sit back and relax on the weekends

For those SRE's that have to work weekends because of a misconfig, rest easy. With Terraform's robust framework, there is very low chance of you working the weekend! You are aware of all the changes a single line of code will do to your infrastructure.

### Pro-Tips:

1. If you need to populate a map with default values, but don't want to use the experimental provider (at the time of writing) -

```hcl
locals {
	map_raw = {
		process_name = "bar"
		ip_cidr = "/27"
	}

	map_filled = {
		for idx, value in local.map_raw: idx => merge(
			{
				resource_limit = "LOW"
				ip_cidr = "/24"
				location = "us-central1-c"
			},
			value
		)
	}
}
```

2. If you have a chain of functions applied to a string, like, upper, replace, split etc, abstract it away into a module
 From:

main.tf

```hcl
locals {
	super_cool_string = upper(replace(split(".", var.value), "-", "_"))
}
```

To:

Module `transform_super_cool_string`

```hcl
variable "value" {
	type = string
}

locals {
	return_val = upper(replace(split(".", var.value), "-", "_"))
}

output "value" {
	value = locals.return_val
}
```

Main.tf

```hcl
module "transformed_string" {
	value = var.value
}

	# Now use module.transformed_string.value
```

Clean, right?

### Conclusion

It was a blast to work with Terraform, especially since their team provides near-instant support over email. It's hard to believe that only v1 of this system is so refined! There are definitely a few places it needs improvement, but for the ordinary user, it should hum along just fine.

Terraform: Lessons Learnt


## Contract Proxy Patterns

> These are the major proxy patterns used in production-grade contracts

### OpenZeppelin Transparent Upgradeable Proxies

**EIP: None**

In transparent proxies, the upgrade of the implementation is handled by the proxy. This leads to a larger contract size for the proxy. It is called "transparent", because the proxy is virtually indistinguishable to the end user from the logic contract (i.e the contract it proxies requests to). There are a few functions which only the admin can access, like `upgradeTo(address implementation)`. None of the requests made by the admin are forwarded to the logic contract, thereby ensuring that [proxy selector clashing](https://medium.com/nomic-labs-blog/malicious-backdoors-in-ethereum-proxies-62629adf3357) does not occur. Note that in most cases, the admin of the proxy will be a contract `ProxyAdmin` .

**Implementation Examples:**

If you're using [hardhat](https://hardhat.org/), you will usually go with one of the following methods -

- Using [hardhat-deploy](https://github.com/wighawag/template-ethereum-contracts/tree/examples/openzeppelin-proxies). Take a look at the contracts and deploy subfolders
- Using [@openzeppelin/hardhat-upgrades](https://www.npmjs.com/package/@openzeppelin/hardhat-upgrades). This [page](https://docs.openzeppelin.com/upgrades-plugins/1.x/#how-plugins-work) has an accurate explanation and run-down on things that happen when you perform an upgrade
- An example you can look at - [upgradeable-contracts-v2](https://github.com/rymnc/upgradeable-contracts-v2)

If you're using [truffle](https://www.trufflesuite.com/truffle), you can use [@openzeppelin/truffle-upgrades](https://www.npmjs.com/package/@openzeppelin/truffle-upgrades)

### OpenZeppelin UUPS

**EIP: [1822](https://eips.ethereum.org/EIPS/eip-1822), [1967](https://eips.ethereum.org/EIPS/eip-1967)**

As mentioned in the EIP, UUPS stands for Universal Upgradeable Proxy Standard

It differs from transparent proxies, by leaving upgrade functionality to the implementation contracts. This allows for smaller proxy size, and cheaper deployment. It uses an ERC1967Proxy implementation for the proxy. By leaving the implementation to handle upgrades, it opens a backdoor of sorts, since the upgrades can be executed by an arbitrary address with selector clashing. That is why it is imperative to setup access-control for the admin functions, or by setting a flag that prevents upgrades.

**Implementation Examples:**

- This [tutorial](https://forum.openzeppelin.com/t/uups-proxies-tutorial-solidity-javascript/7786) by OZ explains the difference between transparent and UUPS proxies, and shows you how to deploy using hardhat and truffle

### Diamond

**EIP: [2535](https://eips.ethereum.org/EIPS/eip-2535)**

The diamond upgrade pattern employs a one-to-many proxy pattern. There can be multiple different implementations, each with their own functions that are proxied to. The best part about this pattern is that all current contracts are compliant to work as "Diamond Facets" (any function that is external to the proxy)

"Facets" can be added/removed/replaced easily, reducing complexity of contracts

Due to the structure of diamond contracts, it can be used to circumvent the 24kb limitation of contract size (since a diamond can have any amount of facets)

It has not been accepted yet, it is being deliberated [here](https://github.com/ethereum/EIPs/issues/2535)

[Nick Mudge](https://dev.to/mudgen) is the author of the pattern, and a lot of resources relating to it are on his blog.

**Implementation Examples:**

- OZ doesn't have a standard diamond library yet, but hardhat-deploy has an example [here](https://github.com/wighawag/hardhat-deploy-test-diamond), which just uses Nick's reference implementation

---

### Conclusion

Contract upgradeability is made easy by the folks over at OZ, which prevent you from reinventing the wheel for standard contracts!

Thanks for reading!


Contract Proxy Patterns

## Introduction

I had some extra time on my hands the past week, and decided to participate in [Gitcoin's GR10 Hackathon](https://gitcoin.co/hackathon/gr10). As I scrolled through the bounties available, [this](https://gitcoin.co/issue/DODOEX/gitCoinGrant/1/100025887) one stood out the most to me. It was right up my alley considering the experience that I have. It also allowed me to work with certain technologies I haven't used before (Example- ViteJS)

### The Deliverable

As mentioned in the problem statement, I had to build an on-chain trading monitor that Dodo could use internally/with end users. They also said that it should be in the form of an SDK, which would be open sourced for anyone to use.

Knowing that each companies infrastructure setup is different, I contacted one of the [team leads](https://github.com/radar-bear) over at Dodo. After a quick call and multiple exchanges over Discord, I was prepared to tackle the project and make sure it cohesively fit into their systems.

### Components of the Deliverable

I split the deliverable into 4 components, namely -

1. The Websocket Engine:

 - This component purely facilitates client side event listening, in a way that most users connect to crypto exchanges, like Binance, Coinbase etc

2. The Monitor SDK

 - This SDK is totally extendable and consists of 4 modules, namely, Sources, Sinks, Middleware and Registries
 - **Sources** are the sources of data, usually it's the evm-based chain. It could also be the mempool for those who wish to execute transactions based on the state of the mempool
 - **Sinks** are the destination for the data. This could be - a Messaging Queue(Kafka, KubeMQ, etc), Email, Slack, Discord, etc.
 - **Middleware** are the pieces of code that fit between the sources and sinks, managing the event flow and transforming data as needed.
 - **Registries** store the on-chain event to real-life implication events. The sources use this to handle user input and convert it to an event it can listen to, onchain

 Note that the middleware can use multiple sources and sinks, and as long as they all
 implement the interfaces designed.

3. The ETH Engine

 - It is highly probable that users will listen to the same events on the same addresses, but with different trigger values(or tolerances).
 - To reduce network load on the node as well as the process, the eth engine smartly manages subscriptions and ensures that there is only one subscription for a given topic.

4. UI

 - Basic Boilerplate for the other DEXes to fork and modify to their need

### Demo

I've yet to work on a full-fledged video demo showcasing the extent of the SDK, but here's a little GIF!


 <img width="80%" src="/assets/demo.svg">


### Conclusion

It was fun to work on this project, it gave me a deeper understanding of typescript's type system as well as sharing types between frontend and backend. Submission date is on July 7th, and results will be announced on July 17th.

Gitcoin GR10 Hackathon


## Introduction

This is going to be a series of posts regarding the development of the project I'm working on, currently unnamed. It will serve as a platform to perform decentralized updates to IoT devices.

There are quite a few pieces I've setup over the week, namely the frontend, backend and contracts as well.

### Frontend

Decided to go with [vitejs](https://vitejs.dev/) for scaffolding a React-Typescript project. Seems more intuitive than CRA so far, I will reserve my opinions for later.

### Backend

Decided to go with [fastify](https://www.fastify.io/), it served me well in the iiot lab miniproject.

- Scaffolded from the [boilerplate repo](https://github.com/rymnc/fastify-ts) I had set up before.
- Integrated [Pinata](https://pinata.cloud/), to pin and unpin device updates on IPFS
- Setup the database with [Prisma](https://www.prisma.io/). The database won't really be holding device information of any sorts, just a link to the ipfs hash which contains device info and is encrypted by the manager's key.
- Looking into auth solutions. Will mostly go with `Auth0`.
- Tests

### Contracts

Decided to go with [hardhat](https://hardhat.org/) as the contract management toolbox. It works great with Typescript.

- Wrote the contract for performing the updates.
- Eventually will split the code up into different contracts, all accessing the same storage.
- This will facilitate upgradeability in the future.
- Not sure of the coverage, but tests have been written.

### Conclusion

The project really needs more team members! Do get in touch with me if you're interested. Primary technologies/languages used will be - Typescript, Solidity, React.

Thanks for reading!


Dev Log #1


## Introduction

I'm currently working on a project that aims to decentralize OTA updates to IoT Devices. However, since OTA updates is a relatively new field for me, I plan to devote some time to research the topic. 🔬

At the time of writing this article, OTA updates are absolutely necessary for IoT devices in terms of security 🔒 and stability. It is essential for the companies to provide after-sales support in terms of updates to their devices for a minimum period of approximately 5 years.

The same statement can be made for any device, but the focus is on IoT devices, which will soon become staple in everyone's homes.

---

### Why are OTA updates necessary?

- Security patches for known vulnerabilities help mitigate attack vectors
- Improves reliability and performance in some cases
- Ensures bug fixes reach all devices that are connected to the internet
- Reduces the need to have technical knowledge to apply the update

---

### Methods of delivering the update

There have been multiple methods to deliver updates to devices, but it seems that it all boils down to three. Organizations will tend to use the method that fits their business needs best.

1. **Edge To Cloud (E2C)**

   The connected device receives the update from a remote server. The received package can either be a hardware or software update. This is an apt solution for organizations which expect only a few devices(which are decoupled in use) per household/network. However this increases network overhead and each device should be able to handle an update on their own.Example - Smartphones📲.

2. **Gateway To Cloud (G2C)**

   Devices that are controlled/operated by a central system (gateway) benefit from this type of delivery. The updates for all devices are received by the gateway from a remote server. The gateway then delivers the updates to the devices. This is a good solution for locations that have multiple devices of the same kind. Network overhead is reduced on the devices as well, since the gateway manages the updates for them. This also provides a single point of failure for the entire ecosystem of devices that it manages. Example - A home security system

3. **Edge To Gateway To Cloud (E2G2C)**

   This architecture consists of a mesh of devices, which can either be edge devices or internet-connected gateways. When an update is received on one of the gateways, it broadcasts it to connected edge devices and other gateways as well. This facilitates easy distribution of updates. It also introduces additional complexity for gateways, and requires them to have a resilient connection to other gateways. Example - Field sensors or Buoys

---

### Existing services that perform OTA updates

1. [AWS IoT](https://docs.aws.amazon.com/freertos/latest/userguide/ota-manager.html)
2. [Upswift](https://www.upswift.io/)
3. [Mender](https://mender.io/)

---

### Advantages of Decentralizing the Distribution

One of the major architectural concerns is the single point of failure that distribution systems suffer from. Sure, you can have a complex internal system that manages it at a high cost, but this doesn't work out too well for startups/smaller organizations that don't nearly have the same financial/engineering capability.

The project I'm working on aims to solve this problem, by leveraging the use of Smart Contracts on Ethereum and decentralized storage on IPFS. Track my progress [here](https://github.com/rymnc/sol-iot-upgrades).

I am also looking for people that can contribute to the project, You can contact me via [LinkedIn](https://www.linkedin.com/in/aaryamannc/)

IoT using blockchain is picking up pace, don't stay in the past!

Thanks for reading!


OTA for IoT


The bread and butter for the network's operation is [devp2p](https://github.com/ethereum/devp2p). Devp2p makes use of a set of network protocols, like libp2p. Unlike libp2p, devp2p was made for the ethereum network, and could be applied for a few similar use-cases as well

There are many implementations of the protocol, as highlighted [here](https://github.com/ethereum/devp2p#implementations)

There are 3 big pieces of devp2p:

1. The Node Record (What can you tell me about your node?)
2. The discovery protocol (How do I find nodes?)
3. The transport protocol (How do I send information to other nodes?)

### 1. The Node Record (What can you tell me about your node?)

- The node record essentially behaves like a node's business card. It tells you it's name (nodeId), where you can find it (IP(v4 & v6) address), and it's role in the organization (the network)

- There are multiple reasons as to why it is important to know all details about the node you are connecting to

- For example, while syncing an archive node, the node would need to connect to other archive nodes to sync state. It can only do this if it knows the type of node on the other end

- The structure of the node record can be found [here](https://github.com/ethereum/devp2p/blob/master/enr.md#record-structure)

- The node signs its record and broadcasts it to other nodes to initiate the connection. The process of signing and encoding the node record can be found [here](https://github.com/ethereum/devp2p/blob/master/enr.md#rlp-encoding)

---

### 2. The discovery protocol (How do I find nodes?)

- Due to the decentralization of routing, each node must maintain its own [DHT](https://docs.ipfs.io/concepts/dht/), that stores node records

- There are various types of packets that are sent to and fro between nodes to establish and secure the connection between them. Technical spec regarding the packets can be found [here](https://github.com/ethereum/devp2p/blob/master/discv4.md#ping-packet-0x01)

- The records are stored in a [Kademlia table](####Kademlia-tables)

- As messages are exchanged between nodes, buckets that store node Ids is filled up from least-recently seen to most-recently seen

- These buckets are DoS resistant due to the design of the bucket structure

- As time progresses, the Nodes DHT starts to fill up with "nearby" nodes, and can easily connect to nodes which are "active"

---

### 3. The transport protocol (How do I send information to other nodes?)

- RLPx is used as the transport protocol to facilitate secure communication between nodes

- Since each node holds a private key, it is used for signing messages that prove its identity, which is highlighted [here](https://github.com/ethereum/devp2p/blob/master/rlpx.md#initial-handshake)

- Framing is a method of encapsulating data that is to be communicated. It is primarily used to support multiplexing multiple protocols over one connection. Technical spec [here](https://github.com/ethereum/devp2p/blob/master/rlpx.md#framing)

- Any message that is sent after the handshake comes with a "capability", which is used to denote the functionality associated with the nodes

- The multiplexing of connections is done based off the message ID, and each capability is given only as much message id space as it needs

---

### Relevant Links

1. [RLPx implementation in Rust](https://docs.rs/rlpx/0.4.1/rlpx/)
2. [Devp2p implementation in Rust](https://github.com/rust-ethereum/sentry)
3. [Whisper Protocol](https://geth.ethereum.org/docs/whisper/whisper-overview)

Thanks for reading!


Notes on devp2p

## Introduction

Usually when starting off with a backend project I'd go with Express and Sequelize. Recently due to a college project I've had the freedom to go with any framework and languages I wanted 💫

Having some Rust experience made learning Typescript enjoyable 😅 (noticed many similarities).

I started looking for a backend framework that would benefit greatly from the typing system, and Fastify was it. It boasts much greater performance than Express as well!

Similarly, I came across Prisma as well. Prisma recently became production-ready, and the typescript support is amazing 🤯

---

### Fastify

Fastify is said to be the fastest Node.js backend framework if used right. That being said, Express could be tuned for performance as well, but it happens out the box for Fastify!

An example of why Fastify is faster, is due to its request-reply schema definitions. Example - If we declared the request body JSON, Fastify leverages `fast-json-stringify` which is vastly faster than the regular `JSON.stringify`. `fast-json-stringify` uses a predefined schema for evaluating JSON bodies.

Developing with Fastify is super easy as well. The route system uses `find-my-way` (which is known to be a "crazy fast HTTP router")

The routes are prefixed with the directory structure, which makes it easy to version APIs, or to eventually split the api into microservices. Example -

```
routes
├── api
│   ├── index.ts
│   ├── v1
│   │   ├── index.ts
│   │   └── users
│   │   ├── index.ts
│   │   └── schemas.ts
│   └── v2
│   ├── index.ts
│   └── users
│   ├── index.ts
│   └── schemas.ts
└── root.ts
```

The routes generated would be - `/api/v1/users/` and `/api/v2/users`. I don't know about you but this feels super intuitive 🚀

Fastify also uses a plugin system with which you can define middleware, routes and decorators.

All this being said, Fastify provides a well rounded experience for everyone!

---

### Prisma

Prisma is an ORM for Node.js/Typescript. What makes it different from Sequelize is that it has typescript support out the box, and is suited for Modern workflows

When compared to Sequelize, Prisma does not have seperate model files for each entity, it has its own schema file which contains definitions for all the models. an Example -

```
model User {
 id Int @id @default(autoincrement())
 createdAt DateTime @default(now())
 updatedAt DateTime @default(now())
 firstName String
 lastName String?
 email String @unique
 CardDetails CardDetails[]
}

model CardDetails {
 id Int @id @default(autoincrement())
 createdAt DateTime @default(now())
 updatedAt DateTime @default(now())
 user User @relation(fields: [userId], references: [id])
 userId Int @unique
 cardNumber String
 cardExpiry DateTime
 cardCvv Int
 cardHolder String
}
```

It is really easy to define models with the [VS Code Extension](https://marketplace.visualstudio.com/items?itemName=Prisma.prisma)

Each time you change this schema file and run the migrate command, Prisma detects the changes and generates the migration. After the migration is run, Prisma generates typings for the new schema as well

Another advantage of Prisma re: migrations is the use of a shadow database to detect schema drift. An explanation I found on their [website](prisma.io)


 <img width="80%" src="/assets/shadow-database.png">


The Prisma docs are verbose, and covers all aspects of using their package. From quickstarts to production!

---

I'm currently using these two technologies for the Industrial IoT Lab Miniproject. Track my progress [here](https://github.com/rymnc/iiot-miniproject)

Thanks for reading!

Experimenting with a new stack

## Introduction

Zero knowledge proofs are cryptographical methods by which one can prove to another that they know a certain value “z” without conveying the actual value “z” to the other person. Understanding them will be easier with the help of an example —

- Alice is color-blind and Bob isn’t

- Alice has two balls, one green and one red. She is unable to tell the difference between the two.

- Alice does not believe that Bob is not color blind

- To verify his claim, Alice holds both balls in her hands and tells Bob that she will either switch/not switch them behind her back, but Bob must tell her if she has switched the balls. If Bob is not color blind, he will be able to tell whether she has switched the balls.

- In the first try of Bob guessing if the ball has switched, he has a 50% chance of guessing it right. Hence, Alice conducts the experiment 20 times in a row. The probability that Bob guessed all of them are right is 9.53674316e^-7. Which is sufficiently small to assume that Bob is, in fact, not colorblind.

- This way, one can confirm the knowledge of the prover without the verifier knowing anything about it.

## History

1. Zero Knowledge Proofs were conceived by 1985 by [Shafi Goldwasser](https://amturing.acm.org/award_winners/goldwasser_8627889.cfm), [Silvio Micali](https://amturing.acm.org/award_winners/micali_9954407.cfm), and [Charles Rackoff ](http://www.cs.toronto.edu/~rackoff/)in their paper “[The Knowledge Complexity of Interactive Proof-Systems](http://scholar.google.co.in/scholar_url?url=https://core.ac.uk/download/pdf/194164868.pdf&hl=en&sa=X&ei=QYRxYJ6SFceSywS5zYnwBg&scisig=AAGBfm3iP2tEPqqwdmm6qMl_rddbqZR8Mg&nossl=1&oi=scholarr)”.

1. Goldwasser, Micali and Rackoff wrote mainly about Interactive Proofs(IP)

1. In 1988, Non-Interactive Zero Knowledge Proofs were conceived by Blum, Feldman, and Micali. They are also known as zk-SNARKs/zk-STARKs.

1. A common reference string shared between the prover and the verifier is sufficient to achieve computational zero-knowledge without requiring interaction. Goldreich and Oren gave impossibility results for one shot zero-knowledge protocols in the standard model. In 2003, Goldwasser and Kalai published an instance of an identification scheme for which any hash function will yield an insecure digital signature scheme.

1. The first widespread application of zk-SNARKs was in the [Zerocash blockchain](https://z.cash/technology/) protocol, where zero-knowledge cryptography provides the computational backbone.

1. It is assumed that By this year, 82 [cryptocurrencies](https://en.wikipedia.org/wiki/Cryptocurrency) worth a total of US$8.85 billion encrypted their transactions with zero-knowledge proofs or a similar private technology.

## Precursory Information

To understand the math behind it, one must have knowledge of Elliptic Curve Cryptography, or ECC in short.

An elliptic curve is the set of points that satisfy a specific mathematical equation. The equation for an elliptic curve looks something like this:

> Example Equation


 <img width="80%" src="/assets/equation.png">


> ECC Visualization


 <img width="80%" src="/assets/ecc.gif">


Elliptic Curves have symmetry about the x axis, and any non-vertical line will intersect the curve in at most three places.

First, we start with an arbitrary point on the curve. Next, we use the dot function to find a new point. We keep repeating the dot function recursively.

If you do this 3 times you may notice that you end at a point E. If you know the starting point, ending point and number of hops it is easy to land at that point. However, if you have no knowledge of the number of hops, it is nearly impossible to arrive at the ending point.

Practically, ECC relies on the existence of a private key and a public key. The private key is used to encrypt the data into a hash, and that hash can be used to verify the owner.

Think of the public key as the starting point and ending point, and the private key as the number of hops.

ECC is important because it provides the same level of encryption as RSA, but with a much reduced bit count.

> _Comparison between RSA and ECC decryption time_


 <img width="80%" src="https://cdn-images-1.medium.com/max/2000/1*RrwkUgYeGZF3GnkF3rmu9g.png">


Quadratic Residue: In number theory, an integer q is called a quadratic residue modulo n if it is congruent to a perfect square modulo n; i.e., if there exists an integer x such that:

> _Quadratic Residue_


 <img width="80%" src="https://cdn-images-1.medium.com/max/2000/1*NhbKASJx4eubOE3raCur4w.png">


A great explanation for quadratic residues is provided by [Polar Pi](https://www.youtube.com/watch?v=34yBBgd_Jpk).

## The Math


 <img width="80%" src="https://cdn-images-1.medium.com/max/2000/1*_Fgowpkx2Xr0eBQNsaVZsg.png">



 <img width="80%" src="https://cdn-images-1.medium.com/max/2000/1*7Q5PK_mHVRMeuYNNkii1Zw.png">


## zk-Proofs and IoT

Internet of Things (IoT) devices are being deployed in huge numbers around the world, and often present serious vulnerabilities. Accordingly, delivering regular software updates is critical to secure IoT devices.

Manufactures face two predominant challenges in providing software updates to IoT devices:

1. scalability of the current client-server model and

2. integrity of the distributed updates — exacerbated due to the devices’ computing power and lightweight cryptographic primitives

_Crowdpatching,_ is a protocol leveraging zk-proofs and Blockchains to deliver security patches to IoT devices.

In this system, the manufacturer delegates the delivery of new updates to self-interested agents, called distributors, who undertake this task in exchange for cryptocurrency (CC) payments.

In this context, the use of a zero-knowledge tool, called zk-SNARKs, is fundamentally important.

Before a proof-of-delivery can be issued, a distributor is expected to provide a zk-SNARKs proof about the provided encrypted update, mathematically proving that: 1) it was indeed obtained encrypting the update file authorized by the manufacturer and 2) the employed key has indeed the provided hash value.

Most importantly, this zero-knowledge proof does not reveal anything else beyond those facts: the unencrypted update and the key remain secret.

If the proof is valid, then the proof-of-delivery signature is generated by the IoT device, and sent to the [Smart Contract](https://ethereum.org/en/glossary/#smart-contract) by the distributor along with the encryption key.

Before issuing the payment, the contract checks that the signature is valid, and that the key matches the hash value. Along with the payment, the SC publishes the key on the BC. The latter can be now used to decrypt the file, and the update is finally obtained by the IoT device.

This was one example of many, on how the strong cryptography protocol of zk-SNARK’s can secure IoT devices.

## Limitations of zk-Proofs

- The computational power required to generate a proof is significantly higher than other protocols.

- The space taken up by said proofs are significantly higher than other protocols as well.

- If the prover loses access to the secret, it is lost forever (this could be an advantage in some cases)

## Current Applications

- Zero Knowledge Proofs are used widely in truly private cryptocurrency transactions, as seen on the prominent L2 solution, **ZK-Rollups**

- _Crowdpatching_

- Highly secure authentication systems

- Securing Intellectual Property

## References

- [https://crypto.stanford.edu/pbc/notes/crypto/zk.html](https://crypto.stanford.edu/pbc/notes/crypto/zk.html)

- [https://eprint.iacr.org/2019/099.pdf](https://eprint.iacr.org/2019/099.pdf)

- [https://core.ac.uk/download/pdf/194164868.pdf](https://core.ac.uk/download/pdf/194164868.pdf)

- [https://www.youtube.com/watch?v=34yBBgd_Jpk](https://www.youtube.com/watch?v=34yBBgd_Jpk)

- [https://arxiv.org/pdf/2010.12134.pdf](https://arxiv.org/pdf/2010.12134.pdf)

- [https://www.cs.umd.edu/~jkatz/papers/bbZK.pdf](https://www.cs.umd.edu/~jkatz/papers/bbZK.pdf)

- [https://github.com/scipr-lab/libsnark](https://github.com/scipr-lab/libsnark)

My notes on the potential of zk-proofs with IoT

Initial notes on ZK-Proofs and IoT


## Introduction

Secret Management for Kubernetes is tricky to implement at times.

There are many avenues for developers working on Kubernetes to access the secrets, if they haven't been setup properly

There are a few good open-source solutions to manage secrets on K8s -

- Hashicorp Vault
- Sealed Secrets
- External Secrets

---

### Hashicorp Vault

As mentioned on their [website](https://vaultproject.io),

> Hashicorp Vault allows Secure, store and tightly control access to tokens, passwords, certificates, encryption keys for protecting secrets and other sensitive data using a UI, CLI, or HTTP API.

Their offering is great, and offers quite a few features that certain teams may find appealing -

- Huge variety of tooling
- Multi-Party signing for Unlocking and Locking the Vault 🔒
- Secret history/rollbacks 📘
- Support for multiple backend data storages, i.e etcd, GCS, etc.
- Robust RBAC and Advanced Permissioning System.

Vault is essentially a one-stop solution for all-things secret management. Hashicorp has a cloud offering as well, to make setup much easier!

There are a few things to keep in mind -

- Unless you're setting it up on their cloud provider it takes time and some technical knowledge.
- If you're using GCP/Azure, you can use their respective secret managers. It works better with their services.
- Expect latency if your Vault is in a different geographical location.

---

### Sealed Secrets

Have you ever wanted to check in your secrets to version control without worrying about it getting compromised?

Or maybe you already checked in secrets accidently 🤦🏽 and got in trouble? Then this solution is for you!

[Sealed Secrets](https://github.com/bitnami-labs/sealed-secrets) is a controller on Kubernetes, which solves a major problem with k8s secret management - Versioning. Sure, it is versioned using the kubeapi server, but for teams that wish to utilize the gitOps approach and have a "hands-off" experience with Kubernetes this is a great option.

Basically, the Operator you create creates an RSA keypair, which is used to encrypt and decrpyt the secrets. Once you create a object of type `SealedSecret`(which is encrypted), the controller automatically creates a Kubernetes `Secret` with the same name, and the desired secret payload.

Here are a few advantages to Sealed Secrets -

- Version Control on Secrets 🤯
- Labels for different "types" of secrets, like certificates, etc
- Tool called `kubeseal` that manages the encryption locally
- Namespace-specific sealing. Also supports cluster-wide secret creation ✅

This product comes from Bitnami, which is a leader in Kubernetes management.

There are a few things to keep in mind -

- Key rotation. They have it documented [here](https://github.com/bitnami-labs/sealed-secrets#sealing-key-renewal)
- Each user that wants to create secrets must have access to the certificate. You would need to implement a distribution mechanism
- No good way to test the integrity of the secret created unless you're an administrator

---

### External Secrets

This is my secret management of choice!

[External Secrets](https://github.com/external-secrets/kubernetes-external-secrets), was produced by GoDaddy to have a generalized secret management interface, with support for multiple backends. This is by far one of the most extensible and flexible secret management options available.

Suppose your organization starts off small. You can use this controller for using the secret manager on whichever cloud you're on. Eventually when you start to scale you may have other ideas regarding secret management and wish to switch to a different provider(Hashicorp Vault, for instance).

The operator has support for multiple backends, like GCP Secret Manager, AWS Secret Manager, Hashicorp Vault, etc. As you may know, each of these services can be modified to suit the usecase as well. The controller will pick up the config from them.

Transitioning from one secret manager to another is super easy. You can pass in your secret-accessor credentials to the controller, and update the configs to the backend that you require. If the secrets exist in the destination manager, they will be picked up! Otherwise the secrets will continue to use the same backend 🚀

Some advantages of External Secrets -

- Support for multiple backend managers
- Automagically manages new secret versions 🧙🏾‍♂️
- Pods can be injected using regular secret volumes
- Tons of examples and assured code quality since the product is by goDaddy
- Can be used with Version Control, since secrets aren't checked into the repo. Only the name is stored. 🙌

There are a few things to keep in mind -

- For secret managers that don't support webhooks, REST calls will be made frequently.
- Secrets must be namespaced

---

This were my views on few of the best secret management systems on Kubernetes. Thanks for reading!


My thoughts on secret management solutions on k8s

Overview of Secret Management on Kubernetes

## Introduction

CI/CD pipelining for Kubernetes is hard 🥱 and messy at times.
There are a couple good CI/CD options out there like Argo CD and Jenkins but they require quite a deal of setting up, especially if you are not familiar with them.

For a team migrating their Monolith to Kubernetes but still want to continue using github actions, this would be a good option. ✅

This is also a good option for small teams(<10) ✅

---

### Requirements

- A Github Account 😉
- Actions enabled on the repository
- Experience with YAML
- A Running GKE Cluster
- Service account with the Kubernetes Engine Admin IAM policy

---

### Initial Setup

1. Get the service account key with the given IAM policy or ask your sys admin to save it in the secrets section of the repo with the name `GKE_CREDENTIALS`

2. Move all your kubernetes manifests into a single directory of choice.

3. Create a workflow.yml in your .github/workflows directory

---

### The Main Course

Copy the following into the workflow.yml

```yaml
name: k8s GitOps
on:
 push:
 branches:
 - "development"
 paths:
 - "k8s/**"
```

What this does is tells the workflow to trigger only when - There is a push to the development branch - There is a change in the k8s/ directory(which is where the manifests live)

#### Get Changed Manifests

In this step we will perform a simple search to find which manifests have been changed. Note that "changed" means either - modified, created, deleted, renamed

1. Checkout the repo

 ```yaml
 get_changed_manifests:
 name: Get changed k8s manifests
 runs-on: ubuntu-latest
 steps:
 - name: checkout
 uses: actions/checkout@v2
 with:
 fetch-depth: 0
 ```

2. Get the list of changed files

 ```yaml
 - name: Get changed files
 id: getfile
 run: |
 echo "::set-output name=files::$(git diff --name-only ${{ github.event.before }}..${{ github.event.after }} | grep 'k8s/' | xargs)"
 ```

> Using `git diff` we are able to get the list of changed files from before the push and after the push

3. Parse the list into a matrix(more on this later)

 ```yaml
 - name: Parse changed files into matrix
 id: matrix
 run: |
 arr=(`echo ${{ steps.getfile.outputs.files }}`);
 empty_array=()
 json_array() {
 echo -n '['
 while [ $# -gt 0 ]; do
 x=${1//\\/\\\\}
 echo -n \"${x//\"/\\\"}\"
 [ $# -gt 1 ] && echo -n ', '
 shift
 done
 echo ']'
 }
 output_arr=$(json_array ${arr[@]})
 output_empty=$(json_array ${empty_array[@]})
 output=$(echo "{\"manifests\": $output_arr}")
 empty=$(echo "{\"manifests\": $output_empty")
 echo "::set-output name=manifests::$output"
 echo "::set-output name=empty_matrix::$empty"
 outputs:
 matrix: ${{ steps.matrix.outputs.manifests }}
 empty_matrix: ${{ steps.matrix.outputs.empty_matrix }}
 ```

> Quite a bit isn't it? What we're doing here is parsing the bash array of changed manifests into a json array. We can then use this json array as a matrix strategy for the next step, i.e to apply these manifests.

> You probably noticed we've also initialized an empty matrix as well. This is used to check whether no files have been changed

#### Verify Changed Manifests

In this step we will be performing a simple sanity check on the structures produced by the previous step

```yaml
check_matrix:
 name: Validate Generated Matrix
 needs: get_changed_manifests
 runs-on: ubuntu-latest
 steps:
 - name: Test with jq
 id: check
 run: |
 matrix='${{ needs.get_changed_manifests.outputs.matrix }}'
 empty_matrix='${{ needs.get_changed_manifests.outputs.empty_matrix }}'
 echo $matrix
 echo $matrix | jq .
 if [[ "$matrix" == "$empty_matrix" ]]; then
 echo "::set-output name=is_empty::true"
 else
 echo "::set-output name=is_empty::false"
 fi
 outputs:
 is_empty: ${{ steps.check.outputs.is_empty }}
```

> Notice that one of the outputs is the `is_empty` variable. We will use this as a conditional for the next step

#### Apply Changed Manifests

In this step we will parse the produced matrix into a strategy and apply/delete the resources as required. Note that ideally we would use `gke-deploy` instead of the regular `kubectl` if it was a package.

1. Check if the changed manifests matrix is empty

 ```yaml
 if: ${{ needs.check_matrix.outputs.is_empty == 'false' }}
 ```

> If it is false the workflow cleanly exits

2. Set Environment Variables

 ```yaml
 env:
 cluster_name: development-cluster
 location: zone-of-cluster
 ```

3. Set the needs and the strategy

 ```yaml
 - name: Apply changed k8s manifests
 runs-on: ubuntu-latest
 needs:
 - get_changed_manifests
 - check_matrix
 strategy:
 matrix: ${{ fromJSON(needs.get_changed_manifests.outputs.matrix) }}
 ```

> `fromJson()` was launched in April, 2020 to make the dev-ex simpler for handling complex workflows. We are using it to parse the json array generated from the first job.

> Putting the previous 2 jobs in the `needs` field ensures that it is pipelined correctly

4. Setup Deployment Scaffold

 ```yaml
 steps:
 - name: Checkout
 uses: actions/checkout@v2
 with:
 fetch-depth: 0
 - name: Get GKE Credentials
 uses: google-github-actions/get-gke-credentials@main
 with:
 cluster_name: ${{ env.cluster_name }}
 location: ${{ env.location }}
 credentials: ${{ secrets.GKE_CREDENTIALS }}
 ```

5. Apply the manifests

 ```yaml
 - name: Apply Manifests
 id: apply
 run: |
 if [[ -f ${{ matrix.manifests }} ]]; then
 kubectl apply -f ${{ matrix.manifests }}
 else
 git show ${{ github.event.before }}:${{ matrix.manifests }} | kubectl delete --ignore-not-found=true -f -
 fi
 ```

> If the file exists, we apply it. If it has been deleted, we get the commit it existed at and delete it from the cluster.

---

That's It! 🚀 A fully custom ci/cd setup for small teams.

Note that you may want to use something more robust if you have hundreds of resources needing to be deployed. This works just fine though! 🎉

---

### Final Notes

Setting this up took quite a bit of time since there were quite a few struggles with the json array bit, but I'm glad it worked out!

Future steps:

- Figure out how to integrate gke-deploy into the last step of the workflow
- Have a workflow that validates manifests in the PR

---

Thanks for reading! 🎉

Setting up GitOps for GKE


## Introduction

I currently work as a Junior Software Engineer at a hybrid fund-operating company, my roles being to develop highly available systems that hook into blockchains.

How did I get here? I often think about this. Guess I was in the right place at the right time 😅

---

### Pilot

The first time I was experienced to code was when my elder sister was working on a school project. I was amazed by how you can write absolute gibberish and it performs the operations as needed!(if you've written it right 😉)

---

### Part One

In the eighth grade we had a choice to choose an elective, and I knew I had to take CS. They started us off with C++(which is crazy in retrospect 😲).

I'm sure none of use knew what we were doing, just copying code off the monitor into our books(and vice-versa). We then moved onto Java which was taught a little more in-depth with concepts such as OOPs covered(which was a solid foundation for things to come 🧱)

---

### Part Two

In High School the focus was shifted to C, a low level language which allows you to break a ton of things if not written correctly! Moved on to SQL and Info-Sec from there

---

### Part Three (Present)

Joined college with Mechatronics Engineering as my Major. We had the opportunity to learn about data structures like graphs, trees, etc, however, it felt like we could have had an extended course on the same. Fast forward to 2020 - **Covid happened 🦠**. We all rushed back home at the earliest.

After being at home stagnant for a couple months I decided to give crypto trading a try, profits were good but I felt something was lacking. I was investing in protocols and chains I knew **nothing** about.

Decided to take up a course on Blockchain on Coursera by UB, SUNY. Was hands down one of the **best** decisions of my life. It introduced me to cryptography, Solidity and the joys of being decentralized.

Fast forward a couple months, and I managed to get my foot in the door with the CTO of the company I currently work at. Started as an intern there, and now a Junior Software Engineer. 🚀

My Stack Now -

      Node.js
      Solidity
      The Ethereum Virtual Machine(EVM)
      SQL/noSQL
      GraphQL
      Kubernetes and Microservices
      GCP
      Docker
      Rust
      Data Analysis in Python(numpy, scipy, pandas, the works)
      React
      Linux Sys-Admin Stuff

Roadmap for 2021 -

      Hasura
      SubGraphs
      Zero Knowledge Proofs
      Roll-Ups on Ethereum

That's my genesis block. 📦


Genesis Block

## Introduction

The [Stealth address](https://eips.ethereum.org/EIPS/eip-5564) scheme was first proposed in 2022, with the [paper](https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10426757) being accepted in 2024. This idea piqued my interest since my work primarily involves [RLN](https://rfc.vac.dev/spec/32/), a modification of [Semaphore](https://semaphore.pse.dev/) which brings rate-limiting capabilities through clever use of the `external_nullifier` used.

While working on RLN, we discovered that when a user registers to the membership set with their public key (more specifically, an `identityCommitment`), they risk deanonymizing their wallet address in the process.

An RLN membership set can be analyzed by the wallet addresses that have called the `register()` function on the smart contract. A need arose for obfuscating these registrations, by having counterparties register identityCommitment's on behalf of other users. This is possible today through the use of gas station networks, however, the privacy aspect of these services can be debated.

The stealth commitment protocol provides a unique mechanism through which Bob can generate an identityCommitment for Alice, and Alice can derive the private key (more specifically, `identitySecret`) for it. Bob can register the commitment on-chain, and signal through the stealth address protocol that someone (Alice) can generate the `identitySecret` for it.

The privacy aspect of the stealth commitment protocol can be further enhanced by using an ephemeral anonymous message passing system, like [Waku](https://waku.org), to ensure that there is plausible deniability that someone participated in the stealthcommitment protocol, and thereby reducing the costs associated with computing pairings on-chain.



## Brief notes on the Stealth Commitment scheme

It is recommended to review the [stealth address](https://eips.ethereum.org/EIPS/eip-5564) EIP before continuing to read through this article! 

## Adapting the Stealth Address protocol for the `alt_bn128` curve

With the vast array of libraries available, computing the stealth commitment is fairly straightforward. 

I chose to implement the stealth commitment protocol in Rust, using [arkworks](https://github.com/arkworks-rs). While the implementation can be adapted for different curves, I implemented it just for the `alt_bn128` curve as a PoC.

The [PoC](https://github.com/rymnc/erc-5564-bn254) can be used in the following way -

```rust
use erc_5564_bn254::{random_keypair, generate_stealth_commitment, generate_stealth_private_key};

fn main() {
 let (spending_key, spending_public_key) = random_keypair();
 let (viewing_key, viewing_public_key) = random_keypair();

 // generate ephemeral keypair
 let (ephemeral_private_key, ephemeral_public_key) = random_keypair();

 let (stealth_commitment, view_tag) = generate_stealth_commitment(viewing_public_key, 
 spending_public_key, 
 ephemeral_private_key);

 let stealth_private_key_opt = generate_stealth_private_key(ephemeral_public_key, 
 viewing_key, 
 spending_key,
 view_tag);

 if stealth_private_key_opt.is_none() {
 panic!("View tags did not match");
 }

 let derived_commitment = derive_public_key(stealth_private_key_opt.unwrap());
 assert_eq!(derived_commitment, stealth_commitment);
}
```

This way, a `stealth_commitment` generated can be registered to the RLN membership set on-chain.



## How Waku can consume the library

The PoC in Rust has C FFI bindings that can be consumed by a variety of languages, including [Nim](https://nim-lang.org), which is used in Waku's primary implementation, [nwaku](https://github.com/waku-org/nwaku). There's an FFI [test suite](https://github.com/rymnc/erc-5564-bn254/blob/9ecb6cf53ce49e638ce0de2e50d06a5e2ed2c487/src/ffi.rs#L269-L316) too!



## Broadcasting Stealth Commitment metadata over Waku

The Stealth Address paper refers to a smart contract that is used for coordination for the `viewing_public_key`, `spending_public_key` and `ephemeral_public_key`. The smart contract emits events which are then received by all users, filtering by valid a `view_tag` from the event log.

One can replace this coordination scheme by using Waku - to broadcast the metadata at regular intervals, which can be received by the intended recipient and processed without any on-chain computation.



Below is the recommended wire format to be used with Waku -

```protobuf
message StealthCommitmentRequest {
	bytes spending_public_key = 1;
	bytes viewing_public_key = 2;
}

message StealthCommitmentPayload {
	bytes stealth_commitment = 1;
	bytes ephemeral_public_key = 2;
}

message StealthCommitmentResponse {
 bytes view_tag = 1;
 bytes stealth_commitment_payload = 3;
}
```

You may notice that the `StealthCommitmentResponse` uses `bytes` for the `stealth_commitment_payload` field. This is recommended since one can partially deserialise the `StealthCommitmentResponse` protobuf to check the `view_tag` , and if it belongs to them, they can proceed to deserialise the `stealth_commitment_payload`. It is faster to do so for commitments that do not belong to a user.



## Stealth Commitments x RLN user flow


 <img width="80%" src="/assets/stealth-commitment-flow.png">


## Future work

Most of the future work is to formalize this into a Waku Stealth Address protocol, which can be used for a varying set of elliptic curves, `alt_bn128` just being one of the supported curves. 

Thanks for reading!

Buffing up your stealth commitment privacy!

Coding Blog

Implementing Stealth Commitments for use in Waku