Cloud-native, event-driven, and serverless architectures are an approach to building and deploying applications that leverage cloud computing resources and services to maximize scalability, efficiency, and flexibility.
This reference architecture demonstrates how Oracle Cloud Infrastructure events and OCI functions can be seamlessly integrated to automatically update Oracle NoSQL Database Cloud Service (Oracle NoSQL) tables when objects are created or modified in an OCI Object Storage bucket. It addresses a common business scenario where data in an Oracle NoSQL table is parsed and updated when files are added or modified in an OCI Object Storage bucket. OCI Events provides a powerful solution for automating tasks based on changes in the state of various cloud resources. It enables the creation of rules that deliver events to specific OCI services, such as OCI Streams**, OCI Functions (as a valuable tool for processing event data and integrating with other OCI services to perform multiple actions), and OCI Notifications. In addition, OCI functions can contain ** to process payload data from OCI events and adjust updates to Oracle NoSQL tables based on specific business needs. This reference architecture provides a comprehensive overview of how to use OCI functions with OCI events to update an Oracle NoSQL table when an object is created or modified in an OCI Object Storage bucket. This reference architecture shows how to use OCI functions and OCI events to update an Oracle NoSQL table when a file is created or updated in OCI Object Storage. The architecture diagram shows the flow of data from the time a file is uploaded to a specified OCI Object Storage bucket. Event service triggers are configured to activate based on specific event rule conditions. The emitted event, in turn, calls a function that extracts data from the uploaded file, does additional processing based on business needs, and ultimately updates the processed data to an Oracle NoSQL table. This approach offers significant advantages by leveraging an event-driven architecture and facilitating real-time data processing. This approach is especially valuable when the data needs to be preprocessed before the target table is updated and the necessary integrity checks are performed to ensure data integrity. Here's how to solve a business use case by parsing and updating data in an Oracle NoSQL table when a file is created or updated in an OCI Object Storage bucket: Set up an event rule in OCI Events to trigger an OCI function when an object is created or updated in an OCI Object Storage bucket.
Create a function to receive events triggered by an event rule. This function handles parsing and updating of data in Oracle NoSQL tables.
Implement the necessary logic within a function to parse the data of an object in an OCI Object Storage bucket.
Interact with Oracle NoSQL Service using the appropriate OCI SDK or API to insert or update records in Oracle NoSQL tables based on parsed data.
By combining OCI events, OCI functions, and Oracle NoSQL Service, we can automate the process of parsing and updating data in Oracle NoSQL tables when files are created or updated in OCI Object Storage. This approach allows for efficient and smooth data processing, as well as synchronization between OCI Object Storage and Oracle NoSQL. The following diagram illustrates this reference architecture. 
Use the OCI console to set up a functional application and create an Oracle NoSQL table.
Collect the following values required to configure the function application environment variables:
User ID The OCID of the user used for authentication.
Tenant ID Tenant OCID. It can be found in the user's profile.
Fingerprints are used to authenticate to OCI APIs.
The region identifier for the region creation request.
API Private Key API Private Key File Location.
Department OCID: Creates the department OCID of the Oracle NoSQL table.
Nosql represents an example of a person table. Create it with the following properties.
The schema consists of the following components:TenantsA tenant is a secure, separate partition in Oracle Cloud that Oracle sets up when you sign up for Oracle Cloud Infrastructure. You can create, organize, and manage resources in Oracle Cloud within your own tenant. A tenant can be synonymous with a company or organization. Typically, a company will have one tenant and its organizational structure will be reflected within that tenant. A tenant is typically associated with a subscription, and a subscription typically has only one tenant. areaAn Oracle Cloud Infrastructure region is a localized geographic region that contains one or more data centers, called availability domains. Regions are independent of each other and far apart (across countries and even continents). PartitioningPartitions are logical separations across regions within an Oracle Cloud Infrastructure tenant. Use partitions to organize resources in Oracle Cloud, control access to resources, and set usage quotas. In order to control access to resources in a specific partition, you need to define policies that specify who can access those resources and what actions they can perform. Availability domainsAn availability domain is an independent, self-contained data center within a region. The physical resources within each availability domain are isolated from the resources of other availability domains, providing fault tolerance. Availability domains don't share infrastructure, such as power or cooling, or an internal network of availability domains. As a result, the failure of one availability domain is unlikely to affect other availability domains within that region. Fault domainsA fault domain is a grouping of hardware and infrastructure within an availability domain. Each availability domain has three fault domains, which have independent power and hardware. When you distribute resources across multiple fault domains, your application can withstand physical server failures, system maintenance, and power failures within the fault domain. Virtual Cloud Networks (VCNs) and SubnetsA VCN is a customizable, software-defined network that you set up in an Oracle Cloud Infrastructure region. Like traditional data center networks, VCNs give you complete control over your network environment. A VCN can have multiple non-overlapping CIDR blocks, which you can change after the VCN is created. You can split a VCN into subnets, which can be within a region or within an availability domain. Each subnet consists of a series of contiguous addresses that do not overlap with other subnets in the VCN. You can change the size of the subnet after it has been created. Subnets can be public or private. Safe listsFor each subnet, you can create security rules that specify the traffic, destination, and type of traffic that must be allowed in and out of the subnet. Services GatewayThe Services Gateway provides access from the VCN to other services, such as Oracle Cloud Infrastructure Object Storage. Traffic from the VCN to Oracle services travels through the Oracle network fabric and does not traverse the Internet. Object storageObject storage provides fast access to large amounts of structured and unstructured data, including database backups, analytics data, and rich content such as images and **. You can securely store your data and then retrieve it directly from within the internet or cloud platform. You can seamlessly scale your storage without experiencing performance degradation or service reliability. For "hot" storage that you need fast, immediate, and frequent access, use standard storage. For "cold" storage that you need to retain for a long time and have little or no access, use archive storage. EventsEvents generated by Oracle Cloud Infrastructure services are structured messages that describe changes to resources. These events cover create, read, update, or delete (CRUD) operations, resource lifecycle state changes, and system events that affect cloud resources. functionsOracle Cloud Infrastructure Functions is a fully managed, multi-tenant, highly scalable, on-demand Functions-as-a-Service (FaaS) platform. It is powered by the FN Project open-source engine. Functions allow you to deploy ** and call it directly or trigger it in response to an event. Oracle Functions uses Docker containers hosted in the Oracle Cloud Infrastructure registry. LogsLogs is a highly scalable and fully managed service that provides access to the following types of logs for resources in the cloud: Audit logs: Logs related to events emitted by the audit service.
Service logs: Logs emitted by individual services such as API Gateway, Events, Functions, Load Balancing, Object Storage, and VCN Flow Logs.
Customize. oracle nosqlOracle NoSQL Database Cloud Service is a fully managed database cloud service designed for database operations that require simple queries with response times in milliseconds. Once you've authenticated your Oracle Cloud account, you can create an Oracle NoSQL table and specify throughput and storage requirements for that table. Oracle retains and manages resources to meet your needs and provide you with the capacity you need. Capacity uses read and write units to specify throughput and GB to specify storage units. The following suggestions can be used as a starting point. Your needs may differ from the architecture described here. vcnWhen you create a VCN, determine the number of CIDR blocks you need and the size of each block based on the number of resources you plan to connect to the VCN subnet. Use CIDR blocks that reside within a standard private IP address space.
Select CIDR blocks that don't overlap with any other networks you intend to have a private connection to (in Oracle Cloud Infrastructure, your on-premises data center, or another cloud provider).
Once you have created a VCN, you can change, add, and remove its CIDR blocks.
When designing subnets, consider where your traffic is going and your security needs. Connect all resources within a specific layer or role to the same subnet, which can act as a security boundary.
Cloud GuardClone and customize the default recipes provided by Oracle to create custom detector and responder recipes. These recipes enable you to specify which types of security violations generate warnings and what actions are allowed to be performed on them. For example, you might want to detect an Object Storage bucket that has visibility set to public.
Apply Cloud Guard at the tenant level to cover the broadest scope and reduce the administrative burden of maintaining multiple configurations.
You can also use the Manage List feature to apply certain configurations to detectors.
functionsThe OCI functions in this architecture are developed using Python. Note that OCI Functions supports a variety of programming languages, and you can develop and deploy functions in the language of your choice. oracle nosqlEnsure that Oracle NoSQL tables are created in the appropriate partitions and that access is managed using OCI Identity and Access Management (IAM) policies. It is important to accurately estimate the read and write capacity before creating a table. Refer to the Oracle documentation and review the recommendations for detailed guidance. In this solution, use a sample table named "person" as an example. Remember to customize the function and table names according to your specific needs. OCI Object StorageThis architecture uses standard object storage to upload files. Make sure to enable emit object events in order to trigger the event service. For sensitive data, private buckets are also recommended. OCI EventsIn this schema, the OCI Events service is configured to listen for Object Creation and Object Update events in OCI Object Storage. The service is called after the object is uploaded to an OCI function and the function is called for processing. When deploying this reference architecture, consider the following points. PerformanceOCI Events, OCI Functions, and Oracle NoSQL are all highly scalable managed services. Oracle NoSQL Service provides on-demand throughput and storage configurations. Before configuring, make sure to estimate the throughput and storage capacity of your Oracle NoSQL Database Cloud Service. SecurityUse policies to restrict who can access OCI resources. For OCI Object Storage, encryption is enabled by default and cannot be turned off. It is recommended to store secret and sensitive data in an OCI vault. Consider using OCI Functions to store API keys, database usernames, passwords, and authentication tokens for authorization with OCI Services. AvailabilityOracle ensures high availability of OCI events, OCI functions, Oracle NoSQL, and OCI logs, all cloud-native and fully managed. Cost: You only pay for the resources that your function uses when it runs. DeploymentThe GitHub repository contains OCI functions that implement this reference architecture. Visit GitHub.
Clone or repositorate to your local computer.
Follow the instructions in the README documentation.
As an Oracle Premier Partner, Agilewing is redefining the way enterprises experience Oracle Cloud Services. With its streamlined account opening process and best-in-class technical support, Agilewing transforms the complex process of account opening and operation into an easy, intuitive experience. With our one-stop shop, you can quickly get up and running with the full range of Oracle Cloud services, so you can seamlessly integrate into the cloud. Agilewing's AgileCDN service, combined with OCI's cloud-based services, provides a best-in-class global content acceleration solution. A strong network of more than 2,800 global POP nodes and 7,000 direct connection points ensures efficient and stable operation no matter where your business expands to the world. Leveraging the advanced technology of Oracle Cloud, Agilewing is committed to simplifying the process of cloud service building, cloud migration, and business going global. "Our partnership model provides customers with cost-effective solutions that allow them to focus more on their core business while enjoying the high performance and security of Oracle Cloud." Oracle Cloud Service, as a promising field, opens the door to new opportunities for enterprises with its high performance, security, and globally consistent service standards. Through Agilewing's professional services, both individual users and enterprises can easily enter this new era full of technological innovation and high performance. Let Agilewing start exploring Oracle Cloud Services and open the door to a whole new world today.