OPiE
Object Programming Environment
Introduction
This writing intends to cover advanced programming concepts that some would consider an extension of the operating system. However, I am including these concepts in a book on programming in order to establish these principles as universally applicable in a computer programming capacity. A more technical description of this paradigm may include hardware-specific features like a description of automata-based data structures and their representation in computer memory. A more high-level approach is presumed in this literature for the sake of limiting the scope of this literature to advanced programming techniques. The paradigm of encapsulating coding hiding the complexity. The premise is analogically the equivalent of “driving” a car but not necessarily knowing how the engine works. These techniques include temporal scheduling, file system operations or object persistence through database storage of serialized objects, user agency and authentication protocols, & networking protocols. The main objective of this software is to provide a “sandbox” programming environment that encapsulates and offers wrapper functionality to many features traditionally provided by the operating system, primarily, the UNIX-based system. The “automata” paradigm of this implementation categorizes it as artificial intelligence while the novel, “nudge” intuition of combining authentication protocols, object serialization, temporal logic, & automata invokes a new paradigm for the power of UNIX command-line based systems. These techniques consist of research performed at the Knowledge Management Lab at Florida International University in Miami, FL, USA and represent advanced programming concepts not instructed in intermediate or advanced college programming courses and is not intended as a deliberation of a completely disseminated functional system, merely a description of the concepts presented. The research was undertaken under the direction of NASA computer engineer Chris Knight of Ames Research Center. Much of this content is taken directly from the development source code repository at the jcrow github account at:
Namely, the OpiE, corevet, objix, and mpy source code repositories.
This literature assumes a working knowledge of programming using examples in python. Also, some materials are included in the appendix for basic UNIX commands and how to implement and use a UNIX OS host on google cloud.
Table of Contents
Object-Orientation
Serialization & Persistence
File I/O
User Agency & Authentication
Temporal Logic
Network Polling
References
User Agency & Authentication
A big issue in modern computing is safeguarding access to data. There are several aspects to security that must be address in modern information systems. Security must be deployed for both internal users of a system and external threats. Authentication defines a paradigm by which access to sensitive data can be secured.
ACL & ACE
The ACL (Access Control List) & ACE (Access Control Entry) authentication protocol is a powerful tool in addressing both internal & external security needs. Basically, the ACL defines a list of ACE’s, each of which defines a set of unix-like permissions on a file or database field which include:
The ACE defines a user or list of users that have a certain type of access to a defined data field and the ACL is a list of ACE entries. Authentication in this case utilizes a username & password mechanism and data fields available in an information system are accessible subject to this protocol.
{graphic}
{code}
import crypt, getpass, pwd
def login(self, p):
username = raw_input('Python login:')
cryptedpasswd = pwd.getpwnam(username)[1]
if cryptedpasswd:
if cryptedpasswd == 'x' or cryptedpasswd == '*':
raise NotImplementedError(
"Sorry, currently no support for shadow passwords")
cleartext = getpass.getpass()
return crypt.crypt(cleartext, cryptedpasswd) == cryptedpasswd
else:
return 1
crypt authenticates by cryptographically converting login password in an impossible to reverse algorithm and matches this encrypted string against the encrypted string in the db.
Here are some example classes implementing the ACL/ACE authentication protocol:
class acl:
__objects = {}
def get_ace(self, objid):
return __objects[objid]
def set_ace(self, objid, permissions=[]):
self.__objects[objid] = permissions
And the ACE:
Object Persistence & Serialization
In more advanced software & operating environments, serialized objects are stored as entries in a database, however, for the purposes of this reading, I am using file i/o operations and strings. Later on I will show an example that combines the concepts of object serialization, user authentication through ACL’s, temporal logic, & object persistence.
import pickle
class Obj:
id = -1
attributes = {}
def get_id(self):
return id
def set_id(self, n):
Id = n
def get_attr(self, key):
return attributes[key]
def set_attr(self, key, val):
attributes[key] = val
def get_all_attr(self):
return attributes
def serialize(self):
return pickle.dumps(self)
r = Obj()
r.set_id = 1
r.set_attr(‘data’, 123)
o = open(‘objfile.dat’, ‘w’)
o.write (r.serialize())
o.close()
Here is the same code with several objects instantiated and stored in a file:
import pickle
class Obj:
id = -1
attributes = {}
def get_id(self):
return id
def set_id(self, n):
Id = n
def get_attr(self, key):
return attributes[key]
def set_attr(self, key, val):
attributes[key] = val
def get_all_attr(self):
return attributes
def serialize(self):
return pickle.dumps(self)
r = Obj()
r.set_id = 1
r.set_attr(‘data’, 123)
r2 = Obj()
r2.set_id = 2
r.set_attr(‘data’, 456)
r3 = Obj()
r3.set_id = 3
r3.set_attr(‘data’, 789(
filedata = ‘’
filedata = r.serialize()+’\n’+r2.serialize()+’\n’+r3.serialize()
o = open(‘objfile2.dat’, ‘w’)
o.write (filedata)
o.close()
A feature of this environment that is useful for data integrity is the execution of function-based inheritance where an update to a parent object automatically loops through and populates each of the new fields in child objects with the added property or object method.
File I/O
Throughout this book, I am using file i/o operations instead of database to avoid including a lot of SQL statements. The necessary file i/o commands are available below.
For reading from a file:
o = open(‘file1.dat’)
filedata = o.read()
print filedata
o.close()
For writing to a file:
O = open(‘file2.dat’, ‘w’)
filedata2 = ‘somedata’
o.write(filedata2)
o.close()
Temporal Logic & Event Management
This paradigm attempts to include time based logical control over a programming code sequence that would be typically considered as part of the operating system, with cron in unix for example. However, I am including this concept in a book on programming to establish this paradigm as an essential conceptual feature in programming in a universal sense.
import time
def func(arg):
print arg
a = time.time()+100
while 1:
If time.time()<a:
time.sleep(30)
else:
func(‘hello world!’)
break
Another interesting variation is to perennially repeat this “timed” function call by adding 60 seconds to the timing feature every time the function is called:
import time
def func(arg):
print arg
a = time.time()+100
while 1:
If time.time()<a:
time.sleep(30)
else:
func(‘hello world!’)
a = a + 60
#break
Also, consider a list of functions each with a time key to be executed temporally:
import time
def func(arg):
print arg
def func2():
print ‘hello world2!’
def func3():
Print ‘hello world3!’
a = time.time()+ 100
b = time.time() + 50
c = time.time() + 125
e = {a: func, b: func2, c: func3}
while 1:
for x in e.keys()
if time.time()<x:
continue
else:
e[x]()
e.pop(x, 0)
if e:
time.sleep(30)
else:
break
Also, using the same code, you can manipulate the event dictionary data structure to execute the listed function at timed intervals:
import time
def func(arg):
print arg
def func2():
print ‘hello world2!’
def func3():
Print ‘hello world3!’
a = time.time()+ 100
b = time.time() + 50
c = time.time() + 125
e = {a: [func, 45], b: [func2, 60], c: [func3, 20]}
while 1:
for x in e.keys()
if time.time()<x:
continue
else:
e[x]()
e[x+e[x][1]] = [e[x][0], e[x][1]]
e.pop(x, 0)
time.sleep(30)
Interoperation between temporal logic and a finite-state machine becomes interesting as time-based execution of functions are arranged along a fsm automata
A critical feature of this temporal logic manager is the premise of an “appointment setter” on a dynamic basis where updates in the data store can generate timed release and response with user interaction through the state machine functionality described below. An interesting add-on based on this concept is a time-based logistics manager that arranges meetings or materials shipment through a cost and temporal API such as SABRE or transit.google.com indicating travel logistics or time-based arrangements resulting in reduced turn-around times. The management of logistics may seem trivial for a small group of people but consider an organization like NASA with thousands of employees. Consider a module that tracks and manages logistics. Notification of changes in flight cost or even drilling down to the maintenance of a vehicle could be invaluable sources of managerial decision-making.
(TODO: edit code so OO db is used, inherit generic Obj class to add state machine properties as in objix, mpy code in cvs repository)
class StateMachine:
def __init__(self):
self.handlers = {}
self.startState = None
self.endStates = []
def add_state(self, name, handler, end_state=0):
name = name.upper()
self.handlers[name] = handler
if end_state:
self.endStates.append(name)
def set_start(self, name):
self.startState = name.upper()
def run(self, cargo):
try:
handler = self.handlers[self.startState]
except:
raise InitializationError("must call .set_start() before .run()")
if not self.endStates:
raise InitializationError("at least one state must be an end_state")
while True:
(newState, cargo) = handler(cargo)
if newState.upper() in self.endStates:
print("reached ", newState)
break
else:
handler = self.handlers[newState.upper()]
while a full-fledged state machine with transitions and temporal logic may appear like this:
def start_transitions(txt):
splitted_txt = txt.split(None,1)
word, txt = splitted_txt if len(splitted_txt) > 1 else (txt,"")
if word == "Python":
newState = "Python_state"
else:
newState = "error_state"
return (newState, txt)
def python_state_transitions(txt):
splitted_txt = txt.split(None,1)
word, txt = splitted_txt if len(splitted_txt) > 1 else (txt,"")
if word == "is":
newState = "is_state"
else:
newState = "error_state"
return (newState, txt)
def is_state_transitions(txt):
splitted_txt = txt.split(None,1)
word, txt = splitted_txt if len(splitted_txt) > 1 else (txt,"")
if word == "not":
newState = "not_state"
elif word in positive_adjectives:
newState = "pos_state"
elif word in negative_adjectives:
newState = "neg_state"
else:
newState = "error_state"
return (newState, txt)
def not_state_transitions(txt):
splitted_txt = txt.split(None,1)
word, txt = splitted_txt if len(splitted_txt) > 1 else (txt,"")
if word in positive_adjectives:
newState = "neg_state"
elif word in negative_adjectives:
newState = "pos_state"
else:
newState = "error_state"
return (newState, txt)
def neg_state(txt):
print("Hallo")
return ("neg_state", "")
if __name__== "__main__":
m = StateMachine()
m.add_state("Start", start_transitions)
m.add_state("Python_state", python_state_transitions)
m.add_state("is_state", is_state_transitions)
m.add_state("not_state", not_state_transitions)
m.add_state("neg_state", None, end_state=1)
m.add_state("pos_state", None, end_state=1)
m.add_state("error_state", None, end_state=1)
m.set_start("Start")
m.run("Python is great")
m.run("Python is difficult")
m.run("Perl is ugly")
Using the example of a decision tree type of finite state machine with the application of project development would appear like this:
{TODO: create statemachine for project diagram: https://www.smartdraw.com/decision-tree/}
TEMPORAL LOGIC, SWITCH OPERATORS, & LOOPS
An interesting approach to encapsulate automata formulas with is by using switch operator functions as loops with a temporal aspect. For example, the previous mention of temporal logic and timing the execution of functions can be used in a process design by re-executing the value of a variable, a sequence repeatable periodically, infinitely or with a finite number of processing.
(Decision Tree)
m = StateMachine()
m
Interestingly, many nodes on this tree would require authentication protocols to other files such as accounting ledgers to determine the budget and project scheduling to decide on timescale. Also, the functions that correspond to transitions between states in this diagram could conveniently be represented visually from an automata model editor populating drop-down lists. An issue with this approach is managing the namespace for these system- or user-defined functions. The likely position of the namespace in memory would be the cron thread or server process loading the python objects. With C++ this position can be detected by pointers however, to retrieve the namespace position in python, for example:
b = obj.__self__
Another interesting application of fsm automata with regards to workflow-based automation of system entry into mainframe applications makes use of the py3270 library offering terminal emulator functions. It is available at:
from py3270 import EmulatorBase
class Emulator(EmulatorBase):
x3270_executable = '/fake/x3270'
s3270_executable = '/fake/s3270'
# use x3270 so you can see what is going on
em = Emulator(visible=True)
# or not (uses s3270)
em = Emulator()
em.connect('3270host.example.com')
em.fill_field(17, 23, 'mylogin', 8)
em.fill_field(18, 23, 'mypass', 8)
em.send_enter()
# if your host unlocks the keyboard before truly being ready you can use:
em.wait_for_field()
# maybe look for a status message
if not em.string_found(1, 2, 'login succesful'):
abort()
# do something useful
# disconnect from host and kill subprocess
em.terminate()
LIBRARY OF CONGRESS EXAMPLE
searching the library of congress mainframe for Harry Potter in Hebrew:
#!/usr/bin/env python
# encoding: utf-8
from PyZ3950 import zoom
from PyZ3950 import zmarc
import sys
reload(sys)
sys.setdefaultencoding('utf8')
conn = zoom.Connection('z3950.loc.gov', 7090)
conn.databaseName = 'VOYAGER'
query = zoom.Query('CCL', 'ti="HARRY POTTER"')
res = conn.search(query)
print "%d hits:" % len(res)
for r in res[:1]:
print unicode( r.data )
https://pypi.python.org/pypi/py3270/0.1.4
Also interesting prospective use is the pywinauto library which can emulate windows application entry & control. It is available at:
a) create an instance of the Application class first and then call start() on it.
>>> from pywinauto import Application
>>> app = Application()
>>> app.start('notepad.exe')
<pywinauto.application.Application object at 0x022991B0>
>>> app.UntitledNotepad.MenuItem("File -> Exit").Select()
b) call the Application.Start() class method. (starts with uppercase 'S')
>>> app = Application.Start('Notepad')
>>> app.UntitledNotepad.MenuItem("File -> Exit").Select()
http://stackoverflow.com/questions/5452830/error-using-pywinauto
https://sourceforge.net/projects/pywinauto/
:
EXECUTING UNIX COMMANDS ON THE WEB
import subprocess
data = cgi.FieldStorage()
if data.has_key('cmd'):
mycmd = data['mycmd'].value
p = subprocess.Popen([, '-a'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
print out
data = cgi.FieldStorage() if data.has_key('ID'): myID = data['myID'].value if testme(ID): #checks if ID exists in dictionary printmessages(myID) addmessage(myID) elif data.has_key('newitem'): newmitem = data['newitem'].value insertmessage(myID, newitem)
Running as a command wrapper for cron or other time-based servers, a bash script can be executed based on timed inputs from an automata FSM scheme
Network Polling
In this code, I am attempting to poll a network of online english-speaking newspapers via http, check the front pages for the appearance of the term “GOLD’, and tally the results. The same can be done with a list of objects, a set of urls, or a list of web service references.
import urllib2
import string
import time
import mysql.connector
import sys
reload(sys)
sys.setdefaultencoding('utf8')
#file i/o operations
f = open('f3.txt')
f = f.read()
f = f.split('\n')
G = 0
for x in f:
try:
p = urllib2.urlopen(x, timeout=30)
p = p.read()
p = string.upper(p)
for y in f2:
if string.find(p, ‘GOLD’)>-1:
G = G + 1
print x+'\t'+y
print x
except:
continue
APPENDIX
USING A GOOGLE CLOUD HOST
https://cloud.google.com/compute/docs/quickstarts
BASIC UNIX COMMANDS
Directories, like folders on a Macintosh, are used to group files together in a hierarchical structure.
You can find out more about these commands by looking up their manpages:
man commandname --- shows you the manual page for the command (UNIX)
UNIX TEXT EDITORS
STARTING PYTHON
>>python
RUNNING PYTHON SCRIPT
>>python filename.py
or:
>>import filename
STARTING UNIX EDITORS
>>nano filename
INSTALLING LAMP ON GOOGLE HOST
https://cloud.google.com/compute/docs/tutorials/setting-up-lamp
SQL
FSM: finite state machine automata info
http://www.python-course.eu/finite_state_machine.php
{TODO: mysql implementation and installation/LAMP, gcloud ftp, .csv}
REFERENCES
Becerra, I., McCarthy, K., & Rodriguez, J. (2001). An Infrastructure for Managing Knowledge Using Intelligent Workflow. Proceedings of the Fourteenth International Florida Artificial Intelligence Research Society Conference, 275-279.
Becerra, I., & Rodriguez, J. (2001). Web Data Mining Techniques for Expertise-Locator Knowledge Management Systems. Proceedings of the Fourteenth International Florida Artificial Intelligence Research Society Conference, 280-285.
Rodriguez, J. (n.d.). jcrow/corevet. Retrieved April 28, 2016, from https://github.com/jcrow/corevet
Rodriguez, J. (n.d.). jcrow/objix. Retrieved April 28, 2016, from https://github.com/jcrow/objix
Rodriguez, J. (n.d.). jcrow/mpy. Retrieved April 28, 2016, from https://github.com/jcrow/mpy
Basic UNIX commands. (n.d.). Retrieved June 23, 2016, from http://mally.stanford.edu/~sr/computing/basic-unix.html
Decision Tree. (n.d.). Retrieved July 10, 2016, from https://www.smartdraw.com/decision-tree/